2024-09-17T03:04:07.392 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-17T03:04:07.413 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T03:04:07.496 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908414 branch: wip-pdonnell-testing-20240916.200549-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/pool-perm} email: pdonnell@redhat.com first_in_suite: false job_id: '7908414' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240916.200549-debug ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: client: pid file: '' mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240916.200549-debug sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 476 sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf sleep_before_teardown: 0 subset: 320/512 suite: fs suite_branch: wip-pdonnell-testing-20240916.200549-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi087.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDEe/9gXlj72LBxMUaRxcRamI5bYmWqTB17Qrk/m225fz+JvPGAnYsBZVhIZGv60j2Tfdf6a9+bgBfWmfYx/ghc= smithi149.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAzVyyv67NP4OhKKq5Ej7UvLDbw+/NEWEzT8Q53yy9Z1BpdB5yWNM2LI1nik3MDr04az/b+66ERM6ibsfxvEFO0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_pool_perm teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_02:09:51 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-17T03:04:07.496 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa; will attempt to use it 2024-09-17T03:04:07.497 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa/tasks 2024-09-17T03:04:07.497 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-17T03:04:07.498 INFO:teuthology.task.internal:Checking packages... 2024-09-17T03:04:07.520 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf' 2024-09-17T03:04:07.520 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-17T03:04:07.520 INFO:teuthology.packaging:ref: None 2024-09-17T03:04:07.520 INFO:teuthology.packaging:tag: None 2024-09-17T03:04:07.520 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240916.200549-debug 2024-09-17T03:04:07.520 INFO:teuthology.packaging:sha1: ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T03:04:07.520 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240916.200549-debug 2024-09-17T03:04:07.661 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4989.gce1b342b 2024-09-17T03:04:07.663 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-17T03:04:07.670 INFO:teuthology.task.internal:no buildpackages task found 2024-09-17T03:04:07.670 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-17T03:04:07.708 INFO:teuthology.task.internal:Saving configuration 2024-09-17T03:04:07.725 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-17T03:04:07.743 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-17T03:04:07.768 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi087.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908414', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 02:59:07.660150', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDEe/9gXlj72LBxMUaRxcRamI5bYmWqTB17Qrk/m225fz+JvPGAnYsBZVhIZGv60j2Tfdf6a9+bgBfWmfYx/ghc='} 2024-09-17T03:04:07.792 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908414', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-17 02:59:07.658291', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAzVyyv67NP4OhKKq5Ej7UvLDbw+/NEWEzT8Q53yy9Z1BpdB5yWNM2LI1nik3MDr04az/b+66ERM6ibsfxvEFO0='} 2024-09-17T03:04:07.793 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-17T03:04:07.801 INFO:teuthology.task.internal:roles: ubuntu@smithi087.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-17T03:04:07.801 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-17T03:04:07.801 INFO:teuthology.run_tasks:Running task console_log... 2024-09-17T03:04:07.885 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9216a83ac0>, signals=[15]) 2024-09-17T03:04:07.885 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-17T03:04:07.891 INFO:teuthology.task.internal:Opening connections... 2024-09-17T03:04:07.891 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi087.front.sepia.ceph.com 2024-09-17T03:04:07.892 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T03:04:07.968 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi149.front.sepia.ceph.com 2024-09-17T03:04:07.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T03:04:08.043 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-17T03:04:08.049 DEBUG:teuthology.orchestra.run.smithi087:> uname -m 2024-09-17T03:04:08.068 INFO:teuthology.orchestra.run.smithi087.stdout:x86_64 2024-09-17T03:04:08.068 DEBUG:teuthology.orchestra.run.smithi087:> cat /etc/os-release 2024-09-17T03:04:08.126 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="CentOS Stream" 2024-09-17T03:04:08.127 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="9" 2024-09-17T03:04:08.127 INFO:teuthology.orchestra.run.smithi087.stdout:ID="centos" 2024-09-17T03:04:08.127 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE="rhel fedora" 2024-09-17T03:04:08.127 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="9" 2024-09-17T03:04:08.127 INFO:teuthology.orchestra.run.smithi087.stdout:PLATFORM_ID="platform:el9" 2024-09-17T03:04:08.127 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T03:04:08.127 INFO:teuthology.orchestra.run.smithi087.stdout:ANSI_COLOR="0;31" 2024-09-17T03:04:08.127 INFO:teuthology.orchestra.run.smithi087.stdout:LOGO="fedora-logo-icon" 2024-09-17T03:04:08.127 INFO:teuthology.orchestra.run.smithi087.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T03:04:08.127 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://centos.org/" 2024-09-17T03:04:08.127 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T03:04:08.127 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T03:04:08.128 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T03:04:08.128 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2024-09-17T03:04:08.152 DEBUG:teuthology.orchestra.run.smithi149:> uname -m 2024-09-17T03:04:08.171 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64 2024-09-17T03:04:08.171 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release 2024-09-17T03:04:08.229 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Stream" 2024-09-17T03:04:08.229 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="9" 2024-09-17T03:04:08.230 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos" 2024-09-17T03:04:08.230 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora" 2024-09-17T03:04:08.230 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="9" 2024-09-17T03:04:08.230 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el9" 2024-09-17T03:04:08.230 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-17T03:04:08.230 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31" 2024-09-17T03:04:08.230 INFO:teuthology.orchestra.run.smithi149.stdout:LOGO="fedora-logo-icon" 2024-09-17T03:04:08.230 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-17T03:04:08.230 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/" 2024-09-17T03:04:08.230 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-17T03:04:08.230 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-17T03:04:08.230 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-17T03:04:08.231 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2024-09-17T03:04:08.252 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-17T03:04:08.260 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-17T03:04:08.285 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-17T03:04:08.285 DEBUG:teuthology.orchestra.run.smithi087:> test '!' -e /home/ubuntu/cephtest 2024-09-17T03:04:08.288 DEBUG:teuthology.orchestra.run.smithi149:> test '!' -e /home/ubuntu/cephtest 2024-09-17T03:04:08.304 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-17T03:04:08.313 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-17T03:04:08.313 DEBUG:teuthology.orchestra.run.smithi087:> test -z $(ls -A /var/lib/ceph) 2024-09-17T03:04:08.346 DEBUG:teuthology.orchestra.run.smithi149:> test -z $(ls -A /var/lib/ceph) 2024-09-17T03:04:08.430 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-17T03:04:08.483 INFO:teuthology.run_tasks:Running task kernel... 2024-09-17T03:04:08.510 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-17T03:04:08.510 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-17T03:04:08.510 DEBUG:teuthology.orchestra.run.smithi087:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T03:04:08.511 DEBUG:teuthology.orchestra.run.smithi149:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-17T03:04:08.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:04:08.528 DEBUG:teuthology.orchestra.run.smithi087:> uname -r 2024-09-17T03:04:08.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:04:08.529 DEBUG:teuthology.orchestra.run.smithi149:> uname -r 2024-09-17T03:04:08.586 INFO:teuthology.orchestra.run.smithi087.stdout:5.14.0-503.el9.x86_64 2024-09-17T03:04:08.586 INFO:teuthology.task.kernel:Running kernel on smithi087: 5.14.0-503.el9.x86_64 2024-09-17T03:04:08.586 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum install -y kernel 2024-09-17T03:04:08.587 INFO:teuthology.orchestra.run.smithi149.stdout:5.14.0-503.el9.x86_64 2024-09-17T03:04:08.587 INFO:teuthology.task.kernel:Running kernel on smithi149: 5.14.0-503.el9.x86_64 2024-09-17T03:04:08.588 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum install -y kernel 2024-09-17T03:04:10.972 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.3 MB 00:00 2024-09-17T03:04:14.049 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:01 2024-09-17T03:04:14.500 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - BaseOS 2.0 MB/s | 8.3 MB 00:04 2024-09-17T03:04:18.965 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - CRB 5.9 MB/s | 6.5 MB 00:01 2024-09-17T03:04:20.676 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - AppStream 4.0 MB/s | 20 MB 00:05 2024-09-17T03:04:21.341 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 19 kB 00:01 2024-09-17T03:04:22.353 INFO:teuthology.orchestra.run.smithi087.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-17T03:04:25.093 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.5 MB 00:00 2024-09-17T03:04:26.577 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - Extras packages 74 kB/s | 19 kB 00:00 2024-09-17T03:04:27.464 INFO:teuthology.orchestra.run.smithi149.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-17T03:04:27.801 INFO:teuthology.orchestra.run.smithi087.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-17T03:04:29.707 INFO:teuthology.orchestra.run.smithi087.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T03:04:29.707 INFO:teuthology.orchestra.run.smithi087.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T03:04:29.818 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2024-09-17T03:04:29.824 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2024-09-17T03:04:29.825 INFO:teuthology.orchestra.run.smithi087.stdout: Package Architecture Version Repository Size 2024-09-17T03:04:29.825 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2024-09-17T03:04:29.825 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2024-09-17T03:04:29.825 INFO:teuthology.orchestra.run.smithi087.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T03:04:29.825 INFO:teuthology.orchestra.run.smithi087.stdout:Installing dependencies: 2024-09-17T03:04:29.825 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T03:04:29.825 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T03:04:29.825 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T03:04:29.825 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:04:29.825 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2024-09-17T03:04:29.825 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2024-09-17T03:04:29.825 INFO:teuthology.orchestra.run.smithi087.stdout:Install 4 Packages 2024-09-17T03:04:29.826 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:04:29.826 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 79 M 2024-09-17T03:04:29.826 INFO:teuthology.orchestra.run.smithi087.stdout:Installed size: 126 M 2024-09-17T03:04:29.826 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2024-09-17T03:04:29.905 INFO:teuthology.orchestra.run.smithi087.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 216.31.2.234) 2024-09-17T03:04:29.905 INFO:teuthology.orchestra.run.smithi087.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 216.31.2.234) 2024-09-17T03:04:29.905 INFO:teuthology.orchestra.run.smithi087.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 216.31.2.234) 2024-09-17T03:04:30.001 INFO:teuthology.orchestra.run.smithi087.stdout:[MIRROR] kernel-modules-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-508.el9.x86_64.rpm (IP: 216.31.2.234) 2024-09-17T03:04:30.007 INFO:teuthology.orchestra.run.smithi087.stdout:[MIRROR] kernel-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-508.el9.x86_64.rpm (IP: 216.31.2.234) 2024-09-17T03:04:30.012 INFO:teuthology.orchestra.run.smithi087.stdout:[MIRROR] kernel-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-508.el9.x86_64.rpm (IP: 216.31.2.234) 2024-09-17T03:04:30.094 INFO:teuthology.orchestra.run.smithi087.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 145 kB/s | 36 kB 00:00 2024-09-17T03:04:30.123 INFO:teuthology.orchestra.run.smithi087.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 216.31.2.234) 2024-09-17T03:04:30.151 INFO:teuthology.orchestra.run.smithi087.stdout:[MIRROR] kernel-modules-core-5.14.0-508.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-508.el9.x86_64.rpm (IP: 216.31.2.234) 2024-09-17T03:04:30.470 INFO:teuthology.orchestra.run.smithi087.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 25 MB/s | 16 MB 00:00 2024-09-17T03:04:30.970 INFO:teuthology.orchestra.run.smithi087.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 31 MB/s | 35 MB 00:01 2024-09-17T03:04:32.937 INFO:teuthology.orchestra.run.smithi149.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-17T03:04:33.353 INFO:teuthology.orchestra.run.smithi087.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 8.8 MB/s | 29 MB 00:03 2024-09-17T03:04:33.356 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2024-09-17T03:04:33.356 INFO:teuthology.orchestra.run.smithi087.stdout:Total 22 MB/s | 79 MB 00:03 2024-09-17T03:04:34.042 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2024-09-17T03:04:34.409 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2024-09-17T03:04:34.409 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2024-09-17T03:04:34.843 INFO:teuthology.orchestra.run.smithi149.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-17T03:04:34.844 INFO:teuthology.orchestra.run.smithi149.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-17T03:04:34.936 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-17T03:04:34.941 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-17T03:04:34.941 INFO:teuthology.orchestra.run.smithi149.stdout: Package Architecture Version Repository Size 2024-09-17T03:04:34.941 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-17T03:04:34.941 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-09-17T03:04:34.941 INFO:teuthology.orchestra.run.smithi149.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-17T03:04:34.942 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-09-17T03:04:34.942 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-17T03:04:34.942 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-17T03:04:34.942 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-17T03:04:34.942 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:04:34.942 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-09-17T03:04:34.942 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-09-17T03:04:34.942 INFO:teuthology.orchestra.run.smithi149.stdout:Install 4 Packages 2024-09-17T03:04:34.942 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:04:34.943 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 79 M 2024-09-17T03:04:34.943 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 126 M 2024-09-17T03:04:34.943 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-09-17T03:04:35.046 INFO:teuthology.orchestra.run.smithi149.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 435 kB/s | 36 kB 00:00 2024-09-17T03:04:35.350 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2024-09-17T03:04:35.350 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2024-09-17T03:04:37.782 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2024-09-17T03:04:38.498 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T03:04:38.572 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T03:04:38.763 INFO:teuthology.orchestra.run.smithi149.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 4.1 MB/s | 16 MB 00:03 2024-09-17T03:04:41.339 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T03:04:41.458 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T03:04:42.396 INFO:teuthology.orchestra.run.smithi149.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 3.9 MB/s | 29 MB 00:07 2024-09-17T03:04:43.854 INFO:teuthology.orchestra.run.smithi149.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 3.9 MB/s | 35 MB 00:08 2024-09-17T03:04:43.855 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-09-17T03:04:43.855 INFO:teuthology.orchestra.run.smithi149.stdout:Total 8.8 MB/s | 79 MB 00:08 2024-09-17T03:04:44.533 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-09-17T03:04:44.899 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-09-17T03:04:44.899 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-09-17T03:04:45.863 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-09-17T03:04:45.863 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-09-17T03:04:46.285 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T03:04:46.333 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T03:04:48.298 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-09-17T03:04:48.955 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-17T03:04:49.038 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T03:04:51.200 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T03:04:51.824 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T03:04:51.966 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T03:04:56.793 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T03:04:56.833 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T03:05:01.675 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T03:05:29.681 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T03:05:29.724 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T03:05:30.060 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T03:05:30.060 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T03:05:30.060 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T03:05:30.060 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T03:05:30.890 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T03:05:30.891 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:05:30.891 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2024-09-17T03:05:30.891 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T03:05:30.891 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T03:05:30.891 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T03:05:30.891 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T03:05:30.891 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:05:30.891 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2024-09-17T03:05:31.236 DEBUG:teuthology.orchestra.run.smithi087:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T03:05:31.320 INFO:teuthology.orchestra.run.smithi087.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T03:05:31.320 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T03:05:31.320 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T03:05:31.320 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T03:05:31.321 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T03:05:31.321 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-17T03:05:31.321 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi087.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T03:05:31.321 DEBUG:teuthology.orchestra.run.smithi087:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T03:05:31.380 INFO:teuthology.orchestra.run.smithi087.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T03:05:31.381 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T03:05:31.381 DEBUG:teuthology.orchestra.run.smithi087:> sudo rpm -qi grub2-tools 2024-09-17T03:05:31.413 INFO:teuthology.orchestra.run.smithi087.stdout:Name : grub2-tools 2024-09-17T03:05:31.413 INFO:teuthology.orchestra.run.smithi087.stdout:Epoch : 1 2024-09-17T03:05:31.413 INFO:teuthology.orchestra.run.smithi087.stdout:Version : 2.06 2024-09-17T03:05:31.413 INFO:teuthology.orchestra.run.smithi087.stdout:Release : 82.el9 2024-09-17T03:05:31.413 INFO:teuthology.orchestra.run.smithi087.stdout:Architecture: x86_64 2024-09-17T03:05:31.413 INFO:teuthology.orchestra.run.smithi087.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T03:05:31.413 INFO:teuthology.orchestra.run.smithi087.stdout:Group : Unspecified 2024-09-17T03:05:31.414 INFO:teuthology.orchestra.run.smithi087.stdout:Size : 8274670 2024-09-17T03:05:31.414 INFO:teuthology.orchestra.run.smithi087.stdout:License : GPLv3+ 2024-09-17T03:05:31.414 INFO:teuthology.orchestra.run.smithi087.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T03:05:31.414 INFO:teuthology.orchestra.run.smithi087.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T03:05:31.414 INFO:teuthology.orchestra.run.smithi087.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T03:05:31.414 INFO:teuthology.orchestra.run.smithi087.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T03:05:31.414 INFO:teuthology.orchestra.run.smithi087.stdout:Packager : builder@centos.org 2024-09-17T03:05:31.414 INFO:teuthology.orchestra.run.smithi087.stdout:Vendor : CentOS 2024-09-17T03:05:31.414 INFO:teuthology.orchestra.run.smithi087.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T03:05:31.414 INFO:teuthology.orchestra.run.smithi087.stdout:Summary : Support tools for GRUB. 2024-09-17T03:05:31.414 INFO:teuthology.orchestra.run.smithi087.stdout:Description : 2024-09-17T03:05:31.414 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:05:31.414 INFO:teuthology.orchestra.run.smithi087.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T03:05:31.414 INFO:teuthology.orchestra.run.smithi087.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T03:05:31.415 INFO:teuthology.orchestra.run.smithi087.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T03:05:31.415 INFO:teuthology.orchestra.run.smithi087.stdout:hardware devices. 2024-09-17T03:05:31.415 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:05:31.415 INFO:teuthology.orchestra.run.smithi087.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T03:05:31.416 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T03:05:31.416 INFO:teuthology.task.kernel:Updating grub on smithi087 to boot 5.14.0-508.el9.x86_64 2024-09-17T03:05:31.416 DEBUG:teuthology.orchestra.run.smithi087:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T03:05:32.026 INFO:teuthology.orchestra.run.smithi087.stderr:Generating grub configuration file ... 2024-09-17T03:05:33.337 INFO:teuthology.orchestra.run.smithi087.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T03:05:33.372 INFO:teuthology.orchestra.run.smithi087.stderr:done 2024-09-17T03:05:33.374 DEBUG:teuthology.orchestra.run.smithi087:> mktemp 2024-09-17T03:05:33.390 INFO:teuthology.orchestra.run.smithi087.stdout:/tmp/tmp.t6XdyE0gsz 2024-09-17T03:05:33.391 DEBUG:teuthology.orchestra.run.smithi087:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.t6XdyE0gsz 2024-09-17T03:05:33.457 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0666 /tmp/tmp.t6XdyE0gsz 2024-09-17T03:05:33.598 DEBUG:teuthology.orchestra.remote:smithi087:/tmp/tmp.t6XdyE0gsz is 6KB 2024-09-17T03:05:33.646 DEBUG:teuthology.orchestra.run.smithi087:> rm -fr /tmp/tmp.t6XdyE0gsz 2024-09-17T03:05:33.661 DEBUG:teuthology.orchestra.run.smithi087:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T03:05:33.730 INFO:teuthology.orchestra.run.smithi087.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T03:05:33.730 INFO:teuthology.orchestra.run.smithi087.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T03:05:33.731 INFO:teuthology.orchestra.run.smithi087.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T03:05:33.731 INFO:teuthology.orchestra.run.smithi087.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T03:05:33.732 DEBUG:teuthology.orchestra.run.smithi087:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T03:05:33.885 DEBUG:teuthology.orchestra.run.smithi087:> sudo shutdown -r now 2024-09-17T03:05:40.008 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T03:05:40.048 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-17T03:05:40.327 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-17T03:05:40.327 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-17T03:05:40.328 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-17T03:05:40.328 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-17T03:05:41.206 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-17T03:05:41.206 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:05:41.206 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-09-17T03:05:41.207 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-17T03:05:41.207 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-17T03:05:41.207 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-17T03:05:41.207 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-17T03:05:41.207 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:05:41.207 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-17T03:05:41.564 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T03:05:41.644 INFO:teuthology.orchestra.run.smithi149.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T03:05:41.644 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T03:05:41.644 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-17T03:05:41.645 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-17T03:05:41.645 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-17T03:05:41.645 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-17T03:05:41.645 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi149.front.sepia.ceph.com, path=None, version=distro) 2024-09-17T03:05:41.645 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q kernel | sort -rV | head -n 1 2024-09-17T03:05:41.704 INFO:teuthology.orchestra.run.smithi149.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-17T03:05:41.705 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-17T03:05:41.705 DEBUG:teuthology.orchestra.run.smithi149:> sudo rpm -qi grub2-tools 2024-09-17T03:05:41.739 INFO:teuthology.orchestra.run.smithi149.stdout:Name : grub2-tools 2024-09-17T03:05:41.740 INFO:teuthology.orchestra.run.smithi149.stdout:Epoch : 1 2024-09-17T03:05:41.740 INFO:teuthology.orchestra.run.smithi149.stdout:Version : 2.06 2024-09-17T03:05:41.740 INFO:teuthology.orchestra.run.smithi149.stdout:Release : 82.el9 2024-09-17T03:05:41.740 INFO:teuthology.orchestra.run.smithi149.stdout:Architecture: x86_64 2024-09-17T03:05:41.740 INFO:teuthology.orchestra.run.smithi149.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-17T03:05:41.740 INFO:teuthology.orchestra.run.smithi149.stdout:Group : Unspecified 2024-09-17T03:05:41.740 INFO:teuthology.orchestra.run.smithi149.stdout:Size : 8274670 2024-09-17T03:05:41.740 INFO:teuthology.orchestra.run.smithi149.stdout:License : GPLv3+ 2024-09-17T03:05:41.740 INFO:teuthology.orchestra.run.smithi149.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-17T03:05:41.740 INFO:teuthology.orchestra.run.smithi149.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-17T03:05:41.740 INFO:teuthology.orchestra.run.smithi149.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-17T03:05:41.741 INFO:teuthology.orchestra.run.smithi149.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-17T03:05:41.741 INFO:teuthology.orchestra.run.smithi149.stdout:Packager : builder@centos.org 2024-09-17T03:05:41.741 INFO:teuthology.orchestra.run.smithi149.stdout:Vendor : CentOS 2024-09-17T03:05:41.741 INFO:teuthology.orchestra.run.smithi149.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-17T03:05:41.741 INFO:teuthology.orchestra.run.smithi149.stdout:Summary : Support tools for GRUB. 2024-09-17T03:05:41.741 INFO:teuthology.orchestra.run.smithi149.stdout:Description : 2024-09-17T03:05:41.741 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:05:41.741 INFO:teuthology.orchestra.run.smithi149.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-17T03:05:41.742 INFO:teuthology.orchestra.run.smithi149.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-17T03:05:41.742 INFO:teuthology.orchestra.run.smithi149.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-17T03:05:41.742 INFO:teuthology.orchestra.run.smithi149.stdout:hardware devices. 2024-09-17T03:05:41.742 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:05:41.742 INFO:teuthology.orchestra.run.smithi149.stdout:This subpackage provides tools for support of all platforms. 2024-09-17T03:05:41.743 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-17T03:05:41.743 INFO:teuthology.task.kernel:Updating grub on smithi149 to boot 5.14.0-508.el9.x86_64 2024-09-17T03:05:41.743 DEBUG:teuthology.orchestra.run.smithi149:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-17T03:05:42.443 INFO:teuthology.orchestra.run.smithi149.stderr:Generating grub configuration file ... 2024-09-17T03:05:43.908 INFO:teuthology.orchestra.run.smithi149.stderr:File descriptor 3 (pipe:[44035]) leaked on vgs invocation. Parent PID 22655: grub2-probe 2024-09-17T03:05:43.909 INFO:teuthology.orchestra.run.smithi149.stderr:File descriptor 9 (pipe:[49400]) leaked on vgs invocation. Parent PID 22655: grub2-probe 2024-09-17T03:05:43.943 INFO:teuthology.orchestra.run.smithi149.stderr:File descriptor 3 (pipe:[44035]) leaked on vgs invocation. Parent PID 22655: grub2-probe 2024-09-17T03:05:43.943 INFO:teuthology.orchestra.run.smithi149.stderr:File descriptor 9 (pipe:[49400]) leaked on vgs invocation. Parent PID 22655: grub2-probe 2024-09-17T03:05:44.107 INFO:teuthology.orchestra.run.smithi149.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-17T03:05:44.131 INFO:teuthology.orchestra.run.smithi149.stderr:done 2024-09-17T03:05:44.133 DEBUG:teuthology.orchestra.run.smithi149:> mktemp 2024-09-17T03:05:44.149 INFO:teuthology.orchestra.run.smithi149.stdout:/tmp/tmp.ue41W9mNA0 2024-09-17T03:05:44.149 DEBUG:teuthology.orchestra.run.smithi149:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ue41W9mNA0 2024-09-17T03:05:44.217 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 0666 /tmp/tmp.ue41W9mNA0 2024-09-17T03:05:44.353 DEBUG:teuthology.orchestra.remote:smithi149:/tmp/tmp.ue41W9mNA0 is 6KB 2024-09-17T03:05:44.403 DEBUG:teuthology.orchestra.run.smithi149:> rm -fr /tmp/tmp.ue41W9mNA0 2024-09-17T03:05:44.419 DEBUG:teuthology.orchestra.run.smithi149:> sudo /bin/ls /boot/loader/entries || true 2024-09-17T03:05:44.482 INFO:teuthology.orchestra.run.smithi149.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-17T03:05:44.482 INFO:teuthology.orchestra.run.smithi149.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-17T03:05:44.482 INFO:teuthology.orchestra.run.smithi149.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-17T03:05:44.482 INFO:teuthology.orchestra.run.smithi149.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-17T03:05:44.484 DEBUG:teuthology.orchestra.run.smithi149:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-17T03:05:44.633 DEBUG:teuthology.orchestra.run.smithi149:> sudo shutdown -r now 2024-09-17T03:06:03.906 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-17T03:06:03.906 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-09-17T03:06:03.908 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T03:06:14.645 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-17T03:06:14.645 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-09-17T03:06:14.646 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T03:06:22.397 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-09-17T03:06:31.403 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-09-17T03:06:31.404 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T03:06:33.149 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-09-17T03:06:34.461 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-09-17T03:06:42.157 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-09-17T03:06:42.157 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T03:06:45.213 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-09-17T03:06:46.463 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-09-17T03:06:46.463 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T03:06:57.225 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-09-17T03:06:57.226 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T03:07:15.645 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-09-17T03:07:18.403 DEBUG:teuthology.orchestra.run.smithi087:> true 2024-09-17T03:07:18.927 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-09-17T03:07:18.928 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-17T03:07:18.928 DEBUG:teuthology.orchestra.run.smithi087:> uname -r 2024-09-17T03:07:18.958 INFO:teuthology.orchestra.run.smithi087.stdout:5.14.0-508.el9.x86_64 2024-09-17T03:07:18.958 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T03:07:18.959 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T03:07:18.959 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T03:07:19.959 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-17T03:07:19.959 DEBUG:teuthology.orchestra.run.smithi087:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T03:07:20.029 INFO:teuthology.orchestra.run.smithi087.stdout:ttyS1 2024-09-17T03:07:20.202 DEBUG:teuthology.parallel:result is None 2024-09-17T03:07:30.646 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-09-17T03:07:30.647 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T03:07:31.028 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-09-17T03:07:31.444 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-09-17T03:07:31.444 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-17T03:07:31.444 DEBUG:teuthology.orchestra.run.smithi149:> uname -r 2024-09-17T03:07:31.459 INFO:teuthology.orchestra.run.smithi149.stdout:5.14.0-508.el9.x86_64 2024-09-17T03:07:31.459 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-17T03:07:31.459 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-17T03:07:31.459 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-17T03:07:32.460 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-17T03:07:32.461 DEBUG:teuthology.orchestra.run.smithi149:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-17T03:07:32.532 INFO:teuthology.orchestra.run.smithi149.stdout:ttyS1 2024-09-17T03:07:32.554 DEBUG:teuthology.parallel:result is None 2024-09-17T03:07:32.554 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-17T03:07:32.562 INFO:teuthology.task.internal:Creating test directory... 2024-09-17T03:07:32.562 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T03:07:32.565 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-17T03:07:32.582 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-17T03:07:32.600 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-17T03:07:32.633 INFO:teuthology.task.internal:Creating archive directory... 2024-09-17T03:07:32.633 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T03:07:32.636 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-17T03:07:32.903 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-17T03:07:32.910 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-17T03:07:32.910 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T03:07:32.913 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-17T03:07:32.941 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T03:07:32.946 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T03:07:32.951 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T03:07:32.958 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-17T03:07:32.960 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-17T03:07:32.975 INFO:teuthology.task.internal:Configuring sudo... 2024-09-17T03:07:32.975 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T03:07:33.003 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-17T03:07:33.028 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-17T03:07:33.038 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-17T03:07:33.038 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T03:07:33.071 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-17T03:07:33.086 DEBUG:teuthology.orchestra.run.smithi087:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T03:07:33.169 DEBUG:teuthology.orchestra.run.smithi087:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T03:07:33.220 DEBUG:teuthology.orchestra.run.smithi087:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T03:07:33.298 DEBUG:teuthology.orchestra.run.smithi087:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T03:07:33.344 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:07:33.344 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T03:07:33.410 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T03:07:33.449 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-17T03:07:33.507 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T03:07:33.585 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-17T03:07:33.624 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:07:33.624 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-17T03:07:33.689 DEBUG:teuthology.orchestra.run.smithi087:> sudo service rsyslog restart 2024-09-17T03:07:33.691 DEBUG:teuthology.orchestra.run.smithi149:> sudo service rsyslog restart 2024-09-17T03:07:33.744 INFO:teuthology.orchestra.run.smithi087.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T03:07:33.774 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T03:07:34.092 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-17T03:07:34.099 INFO:teuthology.task.internal:Starting timer... 2024-09-17T03:07:34.100 INFO:teuthology.run_tasks:Running task pcp... 2024-09-17T03:07:34.155 INFO:teuthology.run_tasks:Running task selinux... 2024-09-17T03:07:34.185 DEBUG:teuthology.orchestra.run.smithi087:> sudo service auditd rotate 2024-09-17T03:07:34.246 INFO:teuthology.orchestra.run.smithi087.stdout:Rotating logs: 2024-09-17T03:07:34.248 DEBUG:teuthology.orchestra.run.smithi149:> sudo service auditd rotate 2024-09-17T03:07:34.341 INFO:teuthology.orchestra.run.smithi149.stdout:Rotating logs: 2024-09-17T03:07:34.343 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-17T03:07:34.343 DEBUG:teuthology.orchestra.run.smithi087:> /usr/sbin/getenforce 2024-09-17T03:07:34.368 INFO:teuthology.orchestra.run.smithi087.stdout:Permissive 2024-09-17T03:07:34.368 DEBUG:teuthology.orchestra.run.smithi149:> /usr/sbin/getenforce 2024-09-17T03:07:34.393 INFO:teuthology.orchestra.run.smithi149.stdout:Permissive 2024-09-17T03:07:34.394 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi087.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi149.front.sepia.ceph.com': 'permissive'} 2024-09-17T03:07:34.394 DEBUG:teuthology.orchestra.run.smithi087:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T03:07:34.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:07:34.439 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T03:07:34.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:07:34.463 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-17T03:07:34.464 DEBUG:teuthology.orchestra.run.smithi087:> sudo /usr/sbin/setenforce permissive 2024-09-17T03:07:34.521 DEBUG:teuthology.orchestra.run.smithi149:> sudo /usr/sbin/setenforce permissive 2024-09-17T03:07:34.570 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-17T03:07:34.581 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-17T03:07:34.593 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-17T03:07:34.594 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi087.front.sepia.ceph.com,smithi149.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-17T03:11:24.887 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi087.front.sepia.ceph.com'), Remote(name='ubuntu@smithi149.front.sepia.ceph.com')] 2024-09-17T03:11:24.888 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-09-17T03:11:24.889 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T03:11:24.978 DEBUG:teuthology.orchestra.run.smithi087:> true 2024-09-17T03:11:25.062 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-09-17T03:11:25.062 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-09-17T03:11:25.063 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-17T03:11:25.139 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-09-17T03:11:25.221 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-09-17T03:11:25.221 INFO:teuthology.run_tasks:Running task clock... 2024-09-17T03:11:25.232 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-17T03:11:25.232 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T03:11:25.233 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T03:11:25.235 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-17T03:11:25.235 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T03:11:25.267 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T03:11:25.284 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T03:11:25.291 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-17T03:11:25.306 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-17T03:11:25.310 INFO:teuthology.orchestra.run.smithi087.stderr:sudo: ntpd: command not found 2024-09-17T03:11:25.321 INFO:teuthology.orchestra.run.smithi087.stdout:506 Cannot talk to daemon 2024-09-17T03:11:25.331 INFO:teuthology.orchestra.run.smithi149.stderr:sudo: ntpd: command not found 2024-09-17T03:11:25.336 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T03:11:25.342 INFO:teuthology.orchestra.run.smithi149.stdout:506 Cannot talk to daemon 2024-09-17T03:11:25.351 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T03:11:25.356 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-17T03:11:25.370 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-17T03:11:25.411 INFO:teuthology.orchestra.run.smithi087.stderr:bash: line 1: ntpq: command not found 2024-09-17T03:11:25.414 INFO:teuthology.orchestra.run.smithi087.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T03:11:25.415 INFO:teuthology.orchestra.run.smithi087.stdout:=============================================================================== 2024-09-17T03:11:25.415 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T03:11:25.415 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T03:11:25.415 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T03:11:25.415 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T03:11:25.436 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-09-17T03:11:25.438 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T03:11:25.438 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-09-17T03:11:25.438 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T03:11:25.438 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T03:11:25.438 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T03:11:25.438 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T03:11:25.439 INFO:teuthology.run_tasks:Running task install... 2024-09-17T03:11:25.447 DEBUG:teuthology.task.install:project ceph 2024-09-17T03:11:25.447 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}} 2024-09-17T03:11:25.447 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'} 2024-09-17T03:11:25.447 INFO:teuthology.task.install:Using flavor: default 2024-09-17T03:11:25.453 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-17T03:11:25.453 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-17T03:11:25.453 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'tag': None, 'wait_for_package': False} 2024-09-17T03:11:25.454 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T03:11:25.456 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'tag': None, 'wait_for_package': False} 2024-09-17T03:11:25.456 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T03:11:25.603 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/centos/9/flavors/default/ 2024-09-17T03:11:25.603 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4989.gce1b342b 2024-09-17T03:11:25.606 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/centos/9/flavors/default/ 2024-09-17T03:11:25.606 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4989.gce1b342b 2024-09-17T03:11:25.886 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T03:11:25.887 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:11:25.887 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T03:11:25.901 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240916.200549-debug/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-17T03:11:25.901 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:11:25.901 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-17T03:11:25.913 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-17T03:11:25.914 DEBUG:teuthology.orchestra.run.smithi149:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T03:11:25.927 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-17T03:11:25.928 DEBUG:teuthology.orchestra.run.smithi087:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ce1b342bf8c6349a8c90cd607f83bff2443f21cf/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-17T03:11:25.980 DEBUG:teuthology.orchestra.run.smithi149:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T03:11:25.994 DEBUG:teuthology.orchestra.run.smithi087:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-17T03:11:26.055 DEBUG:teuthology.orchestra.run.smithi149:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T03:11:26.069 DEBUG:teuthology.orchestra.run.smithi087:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-17T03:11:26.119 INFO:teuthology.orchestra.run.smithi149.stdout:check_obsoletes = 1 2024-09-17T03:11:26.121 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum clean all 2024-09-17T03:11:26.133 INFO:teuthology.orchestra.run.smithi087.stdout:check_obsoletes = 1 2024-09-17T03:11:26.135 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum clean all 2024-09-17T03:11:26.363 INFO:teuthology.orchestra.run.smithi149.stdout:45 files removed 2024-09-17T03:11:26.372 INFO:teuthology.orchestra.run.smithi087.stdout:45 files removed 2024-09-17T03:11:26.391 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-17T03:11:26.399 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-17T03:11:26.949 INFO:teuthology.orchestra.run.smithi149.stdout:ceph packages for x86_64 314 kB/s | 84 kB 00:00 2024-09-17T03:11:26.954 INFO:teuthology.orchestra.run.smithi087.stdout:ceph packages for x86_64 311 kB/s | 84 kB 00:00 2024-09-17T03:11:27.238 INFO:teuthology.orchestra.run.smithi087.stdout:ceph noarch packages 100 kB/s | 24 kB 00:00 2024-09-17T03:11:27.241 INFO:teuthology.orchestra.run.smithi149.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-17T03:11:27.514 INFO:teuthology.orchestra.run.smithi087.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-17T03:11:27.517 INFO:teuthology.orchestra.run.smithi149.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-17T03:11:28.464 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - BaseOS 9.0 MB/s | 8.3 MB 00:00 2024-09-17T03:11:29.478 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.3 MB 00:01 2024-09-17T03:11:30.348 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - AppStream 25 MB/s | 20 MB 00:00 2024-09-17T03:11:32.363 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - AppStream 11 MB/s | 20 MB 00:01 2024-09-17T03:11:37.344 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - CRB 2.0 MB/s | 6.5 MB 00:03 2024-09-17T03:11:38.187 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.5 MB 00:01 2024-09-17T03:11:38.900 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - Extras packages 100 kB/s | 19 kB 00:00 2024-09-17T03:11:39.762 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 19 kB 00:00 2024-09-17T03:11:39.788 INFO:teuthology.orchestra.run.smithi087.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-17T03:11:40.666 INFO:teuthology.orchestra.run.smithi149.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-17T03:11:45.277 INFO:teuthology.orchestra.run.smithi087.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-17T03:11:46.330 INFO:teuthology.orchestra.run.smithi149.stdout:lab-extras 44 kB/s | 1.7 kB 00:00 2024-09-17T03:11:47.480 INFO:teuthology.orchestra.run.smithi087.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-17T03:11:47.481 INFO:teuthology.orchestra.run.smithi087.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-17T03:11:47.494 INFO:teuthology.orchestra.run.smithi087.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-17T03:11:47.495 INFO:teuthology.orchestra.run.smithi087.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-17T03:11:47.496 INFO:teuthology.orchestra.run.smithi087.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-17T03:11:47.497 INFO:teuthology.orchestra.run.smithi087.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-17T03:11:47.499 INFO:teuthology.orchestra.run.smithi087.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-17T03:11:47.502 INFO:teuthology.orchestra.run.smithi087.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-17T03:11:47.503 INFO:teuthology.orchestra.run.smithi087.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-17T03:11:47.566 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2024-09-17T03:11:47.587 INFO:teuthology.orchestra.run.smithi087.stdout:======================================================================================= 2024-09-17T03:11:47.587 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repository Size 2024-09-17T03:11:47.587 INFO:teuthology.orchestra.run.smithi087.stdout:======================================================================================= 2024-09-17T03:11:47.587 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2024-09-17T03:11:47.587 INFO:teuthology.orchestra.run.smithi087.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-17T03:11:47.587 INFO:teuthology.orchestra.run.smithi087.stdout: ceph x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 6.6 k 2024-09-17T03:11:47.588 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-base x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 5.5 M 2024-09-17T03:11:47.588 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-fuse x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 873 k 2024-09-17T03:11:47.588 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 148 k 2024-09-17T03:11:47.588 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 1.5 M 2024-09-17T03:11:47.588 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 154 k 2024-09-17T03:11:47.588 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 14 M 2024-09-17T03:11:47.588 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 7.4 M 2024-09-17T03:11:47.588 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-rook noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 49 k 2024-09-17T03:11:47.588 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-radosgw x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 10 M 2024-09-17T03:11:47.588 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-test x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 40 M 2024-09-17T03:11:47.588 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-volume noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 285 k 2024-09-17T03:11:47.588 INFO:teuthology.orchestra.run.smithi087.stdout: cephadm noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 784 k 2024-09-17T03:11:47.589 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-mirror x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 225 k 2024-09-17T03:11:47.589 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-shell x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 27 k 2024-09-17T03:11:47.589 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-top noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 18 k 2024-09-17T03:11:47.589 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-17T03:11:47.589 INFO:teuthology.orchestra.run.smithi087.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-17T03:11:47.589 INFO:teuthology.orchestra.run.smithi087.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-17T03:11:47.589 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs-devel x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 32 k 2024-09-17T03:11:47.589 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs2 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 766 k 2024-09-17T03:11:47.589 INFO:teuthology.orchestra.run.smithi087.stdout: librados-devel x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 127 k 2024-09-17T03:11:47.589 INFO:teuthology.orchestra.run.smithi087.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-17T03:11:47.589 INFO:teuthology.orchestra.run.smithi087.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-17T03:11:47.589 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-17T03:11:47.590 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-17T03:11:47.590 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-17T03:11:47.590 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-17T03:11:47.590 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cephfs x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 162 k 2024-09-17T03:11:47.590 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rados x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 321 k 2024-09-17T03:11:47.590 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rbd x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 305 k 2024-09-17T03:11:47.590 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rgw x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 99 k 2024-09-17T03:11:47.590 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-fuse x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 86 k 2024-09-17T03:11:47.590 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-mirror x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 3.1 M 2024-09-17T03:11:47.590 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-nbd x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 170 k 2024-09-17T03:11:47.590 INFO:teuthology.orchestra.run.smithi087.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-17T03:11:47.590 INFO:teuthology.orchestra.run.smithi087.stdout:Upgrading: 2024-09-17T03:11:47.591 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-17T03:11:47.591 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-17T03:11:47.591 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-17T03:11:47.591 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-17T03:11:47.591 INFO:teuthology.orchestra.run.smithi087.stdout: librados2 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 3.4 M 2024-09-17T03:11:47.591 INFO:teuthology.orchestra.run.smithi087.stdout: librbd1 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 3.2 M 2024-09-17T03:11:47.591 INFO:teuthology.orchestra.run.smithi087.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-17T03:11:47.591 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-17T03:11:47.591 INFO:teuthology.orchestra.run.smithi087.stdout:Installing dependencies: 2024-09-17T03:11:47.591 INFO:teuthology.orchestra.run.smithi087.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T03:11:47.591 INFO:teuthology.orchestra.run.smithi087.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-17T03:11:47.591 INFO:teuthology.orchestra.run.smithi087.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T03:11:47.591 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-common x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 20 M 2024-09-17T03:11:47.592 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 34 k 2024-09-17T03:11:47.592 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mds x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 2.3 M 2024-09-17T03:11:47.592 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 291 k 2024-09-17T03:11:47.592 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mon x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 5.0 M 2024-09-17T03:11:47.592 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-osd x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 18 M 2024-09-17T03:11:47.592 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 16 k 2024-09-17T03:11:47.592 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-selinux x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 25 k 2024-09-17T03:11:47.592 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T03:11:47.592 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T03:11:47.592 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T03:11:47.592 INFO:teuthology.orchestra.run.smithi087.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T03:11:47.593 INFO:teuthology.orchestra.run.smithi087.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T03:11:47.593 INFO:teuthology.orchestra.run.smithi087.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T03:11:47.593 INFO:teuthology.orchestra.run.smithi087.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T03:11:47.593 INFO:teuthology.orchestra.run.smithi087.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T03:11:47.593 INFO:teuthology.orchestra.run.smithi087.stdout: libcephsqlite x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 159 k 2024-09-17T03:11:47.593 INFO:teuthology.orchestra.run.smithi087.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T03:11:47.593 INFO:teuthology.orchestra.run.smithi087.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-17T03:11:47.593 INFO:teuthology.orchestra.run.smithi087.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T03:11:47.593 INFO:teuthology.orchestra.run.smithi087.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T03:11:47.593 INFO:teuthology.orchestra.run.smithi087.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T03:11:47.593 INFO:teuthology.orchestra.run.smithi087.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T03:11:47.593 INFO:teuthology.orchestra.run.smithi087.stdout: libradosstriper1 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 507 k 2024-09-17T03:11:47.593 INFO:teuthology.orchestra.run.smithi087.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T03:11:47.594 INFO:teuthology.orchestra.run.smithi087.stdout: librgw2 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 5.3 M 2024-09-17T03:11:47.594 INFO:teuthology.orchestra.run.smithi087.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T03:11:47.594 INFO:teuthology.orchestra.run.smithi087.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-17T03:11:47.594 INFO:teuthology.orchestra.run.smithi087.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T03:11:47.594 INFO:teuthology.orchestra.run.smithi087.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T03:11:47.594 INFO:teuthology.orchestra.run.smithi087.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T03:11:47.594 INFO:teuthology.orchestra.run.smithi087.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T03:11:47.594 INFO:teuthology.orchestra.run.smithi087.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T03:11:47.594 INFO:teuthology.orchestra.run.smithi087.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T03:11:47.594 INFO:teuthology.orchestra.run.smithi087.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T03:11:47.594 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-17T03:11:47.594 INFO:teuthology.orchestra.run.smithi087.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T03:11:47.594 INFO:teuthology.orchestra.run.smithi087.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T03:11:47.595 INFO:teuthology.orchestra.run.smithi087.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T03:11:47.595 INFO:teuthology.orchestra.run.smithi087.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-17T03:11:47.595 INFO:teuthology.orchestra.run.smithi087.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T03:11:47.595 INFO:teuthology.orchestra.run.smithi087.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T03:11:47.595 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T03:11:47.595 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-argparse x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 45 k 2024-09-17T03:11:47.595 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-common x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 143 k 2024-09-17T03:11:47.595 INFO:teuthology.orchestra.run.smithi087.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T03:11:47.595 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T03:11:47.595 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T03:11:47.595 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T03:11:47.595 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-17T03:11:47.595 INFO:teuthology.orchestra.run.smithi087.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-17T03:11:47.595 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T03:11:47.596 INFO:teuthology.orchestra.run.smithi087.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T03:11:47.596 INFO:teuthology.orchestra.run.smithi087.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T03:11:47.596 INFO:teuthology.orchestra.run.smithi087.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T03:11:47.596 INFO:teuthology.orchestra.run.smithi087.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T03:11:47.596 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T03:11:47.596 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T03:11:47.596 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-17T03:11:47.596 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T03:11:47.596 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T03:11:47.596 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T03:11:47.596 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T03:11:47.596 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T03:11:47.596 INFO:teuthology.orchestra.run.smithi087.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T03:11:47.597 INFO:teuthology.orchestra.run.smithi087.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T03:11:47.597 INFO:teuthology.orchestra.run.smithi087.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T03:11:47.597 INFO:teuthology.orchestra.run.smithi087.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T03:11:47.597 INFO:teuthology.orchestra.run.smithi087.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T03:11:47.597 INFO:teuthology.orchestra.run.smithi087.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T03:11:47.597 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T03:11:47.597 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T03:11:47.597 INFO:teuthology.orchestra.run.smithi087.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T03:11:47.597 INFO:teuthology.orchestra.run.smithi087.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T03:11:47.597 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T03:11:47.597 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T03:11:47.597 INFO:teuthology.orchestra.run.smithi087.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T03:11:47.597 INFO:teuthology.orchestra.run.smithi087.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T03:11:47.597 INFO:teuthology.orchestra.run.smithi087.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T03:11:47.598 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T03:11:47.598 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T03:11:47.598 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T03:11:47.598 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T03:11:47.598 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-17T03:11:47.598 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T03:11:47.598 INFO:teuthology.orchestra.run.smithi087.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T03:11:47.598 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T03:11:47.598 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T03:11:47.598 INFO:teuthology.orchestra.run.smithi087.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T03:11:47.598 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T03:11:47.598 INFO:teuthology.orchestra.run.smithi087.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T03:11:47.598 INFO:teuthology.orchestra.run.smithi087.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T03:11:47.599 INFO:teuthology.orchestra.run.smithi087.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T03:11:47.599 INFO:teuthology.orchestra.run.smithi087.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T03:11:47.599 INFO:teuthology.orchestra.run.smithi087.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T03:11:47.599 INFO:teuthology.orchestra.run.smithi087.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-17T03:11:47.599 INFO:teuthology.orchestra.run.smithi087.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T03:11:47.599 INFO:teuthology.orchestra.run.smithi087.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T03:11:47.599 INFO:teuthology.orchestra.run.smithi087.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T03:11:47.599 INFO:teuthology.orchestra.run.smithi087.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T03:11:47.599 INFO:teuthology.orchestra.run.smithi087.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T03:11:47.599 INFO:teuthology.orchestra.run.smithi087.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T03:11:47.599 INFO:teuthology.orchestra.run.smithi087.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T03:11:47.599 INFO:teuthology.orchestra.run.smithi087.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T03:11:47.599 INFO:teuthology.orchestra.run.smithi087.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T03:11:47.599 INFO:teuthology.orchestra.run.smithi087.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-17T03:11:47.600 INFO:teuthology.orchestra.run.smithi087.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T03:11:47.600 INFO:teuthology.orchestra.run.smithi087.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-17T03:11:47.600 INFO:teuthology.orchestra.run.smithi087.stdout:Installing weak dependencies: 2024-09-17T03:11:47.600 INFO:teuthology.orchestra.run.smithi087.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T03:11:47.600 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:11:47.600 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2024-09-17T03:11:47.600 INFO:teuthology.orchestra.run.smithi087.stdout:======================================================================================= 2024-09-17T03:11:47.600 INFO:teuthology.orchestra.run.smithi087.stdout:Install 149 Packages 2024-09-17T03:11:47.600 INFO:teuthology.orchestra.run.smithi087.stdout:Upgrade 8 Packages 2024-09-17T03:11:47.600 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:11:47.601 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 228 M 2024-09-17T03:11:47.601 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2024-09-17T03:11:48.278 INFO:teuthology.orchestra.run.smithi087.stdout:(1/157): ceph-19.3.0-4989.gce1b342b.el9.x86_64. 59 kB/s | 6.6 kB 00:00 2024-09-17T03:11:48.462 INFO:teuthology.orchestra.run.smithi087.stdout:(2/157): ceph-fuse-19.3.0-4989.gce1b342b.el9.x8 4.6 MB/s | 873 kB 00:00 2024-09-17T03:11:48.580 INFO:teuthology.orchestra.run.smithi087.stdout:(3/157): ceph-immutable-object-cache-19.3.0-498 1.6 MB/s | 148 kB 00:00 2024-09-17T03:11:48.581 INFO:teuthology.orchestra.run.smithi149.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-17T03:11:48.581 INFO:teuthology.orchestra.run.smithi149.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-17T03:11:48.581 INFO:teuthology.orchestra.run.smithi149.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-17T03:11:48.581 INFO:teuthology.orchestra.run.smithi149.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-17T03:11:48.581 INFO:teuthology.orchestra.run.smithi149.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-17T03:11:48.581 INFO:teuthology.orchestra.run.smithi149.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-17T03:11:48.581 INFO:teuthology.orchestra.run.smithi149.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-17T03:11:48.581 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-17T03:11:48.581 INFO:teuthology.orchestra.run.smithi149.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-17T03:11:48.634 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-09-17T03:11:48.635 INFO:teuthology.orchestra.run.smithi087.stdout:(4/157): ceph-base-19.3.0-4989.gce1b342b.el9.x8 12 MB/s | 5.5 MB 00:00 2024-09-17T03:11:48.638 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-09-17T03:11:48.638 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-09-17T03:11:48.638 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-09-17T03:11:48.638 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-09-17T03:11:48.638 INFO:teuthology.orchestra.run.smithi149.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-17T03:11:48.638 INFO:teuthology.orchestra.run.smithi149.stdout: ceph x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 6.6 k 2024-09-17T03:11:48.638 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 5.5 M 2024-09-17T03:11:48.639 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 873 k 2024-09-17T03:11:48.639 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 148 k 2024-09-17T03:11:48.639 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 1.5 M 2024-09-17T03:11:48.639 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 154 k 2024-09-17T03:11:48.639 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 14 M 2024-09-17T03:11:48.639 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 7.4 M 2024-09-17T03:11:48.639 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 49 k 2024-09-17T03:11:48.639 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 10 M 2024-09-17T03:11:48.639 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 40 M 2024-09-17T03:11:48.639 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 285 k 2024-09-17T03:11:48.639 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 784 k 2024-09-17T03:11:48.639 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 225 k 2024-09-17T03:11:48.639 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-shell x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 27 k 2024-09-17T03:11:48.640 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 18 k 2024-09-17T03:11:48.640 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-17T03:11:48.640 INFO:teuthology.orchestra.run.smithi149.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-17T03:11:48.640 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-17T03:11:48.640 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 32 k 2024-09-17T03:11:48.640 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 766 k 2024-09-17T03:11:48.640 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 127 k 2024-09-17T03:11:48.640 INFO:teuthology.orchestra.run.smithi149.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-17T03:11:48.640 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-17T03:11:48.640 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-17T03:11:48.640 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-17T03:11:48.640 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-17T03:11:48.640 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-17T03:11:48.641 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 162 k 2024-09-17T03:11:48.641 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 321 k 2024-09-17T03:11:48.641 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 305 k 2024-09-17T03:11:48.641 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 99 k 2024-09-17T03:11:48.641 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 86 k 2024-09-17T03:11:48.641 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 3.1 M 2024-09-17T03:11:48.641 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 170 k 2024-09-17T03:11:48.641 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-17T03:11:48.641 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrading: 2024-09-17T03:11:48.641 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-17T03:11:48.642 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-17T03:11:48.642 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-17T03:11:48.642 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-17T03:11:48.642 INFO:teuthology.orchestra.run.smithi149.stdout: librados2 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 3.4 M 2024-09-17T03:11:48.642 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 3.2 M 2024-09-17T03:11:48.642 INFO:teuthology.orchestra.run.smithi149.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-17T03:11:48.642 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-17T03:11:48.642 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-09-17T03:11:48.642 INFO:teuthology.orchestra.run.smithi149.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-17T03:11:48.642 INFO:teuthology.orchestra.run.smithi149.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-17T03:11:48.642 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-17T03:11:48.643 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 20 M 2024-09-17T03:11:48.643 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 34 k 2024-09-17T03:11:48.643 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 2.3 M 2024-09-17T03:11:48.643 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 291 k 2024-09-17T03:11:48.643 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 5.0 M 2024-09-17T03:11:48.643 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 18 M 2024-09-17T03:11:48.643 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4989.gce1b342b.el9 ceph-noarch 16 k 2024-09-17T03:11:48.643 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 25 k 2024-09-17T03:11:48.643 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-17T03:11:48.643 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-17T03:11:48.643 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-17T03:11:48.643 INFO:teuthology.orchestra.run.smithi149.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-17T03:11:48.643 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-17T03:11:48.644 INFO:teuthology.orchestra.run.smithi149.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-17T03:11:48.644 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-17T03:11:48.644 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-17T03:11:48.644 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 159 k 2024-09-17T03:11:48.644 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-17T03:11:48.644 INFO:teuthology.orchestra.run.smithi149.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-17T03:11:48.644 INFO:teuthology.orchestra.run.smithi149.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-17T03:11:48.644 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-17T03:11:48.645 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-17T03:11:48.645 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-17T03:11:48.645 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 507 k 2024-09-17T03:11:48.645 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-17T03:11:48.645 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2 x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 5.3 M 2024-09-17T03:11:48.645 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-17T03:11:48.645 INFO:teuthology.orchestra.run.smithi149.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-17T03:11:48.645 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-17T03:11:48.645 INFO:teuthology.orchestra.run.smithi149.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-17T03:11:48.645 INFO:teuthology.orchestra.run.smithi149.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-17T03:11:48.645 INFO:teuthology.orchestra.run.smithi149.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-17T03:11:48.645 INFO:teuthology.orchestra.run.smithi149.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-17T03:11:48.645 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-17T03:11:48.646 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-17T03:11:48.646 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-17T03:11:48.646 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-17T03:11:48.646 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-17T03:11:48.646 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-17T03:11:48.646 INFO:teuthology.orchestra.run.smithi149.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-17T03:11:48.646 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-17T03:11:48.646 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-17T03:11:48.646 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-17T03:11:48.646 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 45 k 2024-09-17T03:11:48.646 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common x86_64 2:19.3.0-4989.gce1b342b.el9 ceph 143 k 2024-09-17T03:11:48.646 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-17T03:11:48.646 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-17T03:11:48.647 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-17T03:11:48.647 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-17T03:11:48.647 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-17T03:11:48.647 INFO:teuthology.orchestra.run.smithi149.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-17T03:11:48.647 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-17T03:11:48.647 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-17T03:11:48.647 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-17T03:11:48.647 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-17T03:11:48.647 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-17T03:11:48.647 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-17T03:11:48.647 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-17T03:11:48.647 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-17T03:11:48.647 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-17T03:11:48.648 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-17T03:11:48.648 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-17T03:11:48.648 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-17T03:11:48.648 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-17T03:11:48.648 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-17T03:11:48.648 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-17T03:11:48.648 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-17T03:11:48.648 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-17T03:11:48.648 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-17T03:11:48.648 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-17T03:11:48.648 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-17T03:11:48.648 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-17T03:11:48.649 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-17T03:11:48.649 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-17T03:11:48.649 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-17T03:11:48.649 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-17T03:11:48.649 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-17T03:11:48.649 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-17T03:11:48.649 INFO:teuthology.orchestra.run.smithi149.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-17T03:11:48.649 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-17T03:11:48.649 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-17T03:11:48.649 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-17T03:11:48.649 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-17T03:11:48.649 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-17T03:11:48.650 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-17T03:11:48.650 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-17T03:11:48.650 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-17T03:11:48.650 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-17T03:11:48.650 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-17T03:11:48.650 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-17T03:11:48.650 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-17T03:11:48.650 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-17T03:11:48.650 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-17T03:11:48.650 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-17T03:11:48.650 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-17T03:11:48.650 INFO:teuthology.orchestra.run.smithi149.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-17T03:11:48.650 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-17T03:11:48.651 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-17T03:11:48.651 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-17T03:11:48.651 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-17T03:11:48.651 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-17T03:11:48.651 INFO:teuthology.orchestra.run.smithi149.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-17T03:11:48.651 INFO:teuthology.orchestra.run.smithi149.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-17T03:11:48.651 INFO:teuthology.orchestra.run.smithi149.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-17T03:11:48.651 INFO:teuthology.orchestra.run.smithi149.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-17T03:11:48.651 INFO:teuthology.orchestra.run.smithi149.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-17T03:11:48.651 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-17T03:11:48.651 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-17T03:11:48.651 INFO:teuthology.orchestra.run.smithi149.stdout:Installing weak dependencies: 2024-09-17T03:11:48.651 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-17T03:11:48.652 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:11:48.652 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-09-17T03:11:48.652 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-09-17T03:11:48.652 INFO:teuthology.orchestra.run.smithi149.stdout:Install 149 Packages 2024-09-17T03:11:48.652 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrade 8 Packages 2024-09-17T03:11:48.652 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:11:48.652 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 228 M 2024-09-17T03:11:48.653 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-09-17T03:11:48.738 INFO:teuthology.orchestra.run.smithi087.stdout:(5/157): ceph-mgr-19.3.0-4989.gce1b342b.el9.x86 14 MB/s | 1.5 MB 00:00 2024-09-17T03:11:48.790 INFO:teuthology.orchestra.run.smithi087.stdout:(6/157): ceph-mds-19.3.0-4989.gce1b342b.el9.x86 9.8 MB/s | 2.3 MB 00:00 2024-09-17T03:11:49.030 INFO:teuthology.orchestra.run.smithi087.stdout:(7/157): ceph-mon-19.3.0-4989.gce1b342b.el9.x86 17 MB/s | 5.0 MB 00:00 2024-09-17T03:11:49.107 INFO:teuthology.orchestra.run.smithi149.stdout:(1/157): ceph-19.3.0-4989.gce1b342b.el9.x86_64. 43 kB/s | 6.6 kB 00:00 2024-09-17T03:11:49.264 INFO:teuthology.orchestra.run.smithi087.stdout:(8/157): ceph-common-19.3.0-4989.gce1b342b.el9. 18 MB/s | 20 MB 00:01 2024-09-17T03:11:49.347 INFO:teuthology.orchestra.run.smithi087.stdout:(9/157): ceph-selinux-19.3.0-4989.gce1b342b.el9 302 kB/s | 25 kB 00:00 2024-09-17T03:11:49.956 INFO:teuthology.orchestra.run.smithi087.stdout:(10/157): ceph-osd-19.3.0-4989.gce1b342b.el9.x8 15 MB/s | 18 MB 00:01 2024-09-17T03:11:49.965 INFO:teuthology.orchestra.run.smithi149.stdout:(2/157): ceph-fuse-19.3.0-4989.gce1b342b.el9.x8 1.0 MB/s | 873 kB 00:00 2024-09-17T03:11:50.014 INFO:teuthology.orchestra.run.smithi087.stdout:(11/157): cephfs-mirror-19.3.0-4989.gce1b342b.e 3.8 MB/s | 225 kB 00:00 2024-09-17T03:11:50.089 INFO:teuthology.orchestra.run.smithi087.stdout:(12/157): cephfs-shell-19.3.0-4989.gce1b342b.el 361 kB/s | 27 kB 00:00 2024-09-17T03:11:50.157 INFO:teuthology.orchestra.run.smithi087.stdout:(13/157): libcephfs-devel-19.3.0-4989.gce1b342b 476 kB/s | 32 kB 00:00 2024-09-17T03:11:50.232 INFO:teuthology.orchestra.run.smithi087.stdout:(14/157): libcephfs2-19.3.0-4989.gce1b342b.el9. 10 MB/s | 766 kB 00:00 2024-09-17T03:11:50.232 INFO:teuthology.orchestra.run.smithi149.stdout:(3/157): ceph-common-19.3.0-4989.gce1b342b.el9. 16 MB/s | 20 MB 00:01 2024-09-17T03:11:50.257 INFO:teuthology.orchestra.run.smithi149.stdout:(4/157): ceph-immutable-object-cache-19.3.0-498 507 kB/s | 148 kB 00:00 2024-09-17T03:11:50.307 INFO:teuthology.orchestra.run.smithi087.stdout:(15/157): libcephsqlite-19.3.0-4989.gce1b342b.e 2.1 MB/s | 159 kB 00:00 2024-09-17T03:11:50.399 INFO:teuthology.orchestra.run.smithi087.stdout:(16/157): librados-devel-19.3.0-4989.gce1b342b. 1.4 MB/s | 127 kB 00:00 2024-09-17T03:11:50.424 INFO:teuthology.orchestra.run.smithi149.stdout:(5/157): ceph-mds-19.3.0-4989.gce1b342b.el9.x86 12 MB/s | 2.3 MB 00:00 2024-09-17T03:11:50.466 INFO:teuthology.orchestra.run.smithi087.stdout:(17/157): libradosstriper1-19.3.0-4989.gce1b342 7.4 MB/s | 507 kB 00:00 2024-09-17T03:11:50.683 INFO:teuthology.orchestra.run.smithi087.stdout:(18/157): ceph-radosgw-19.3.0-4989.gce1b342b.el 6.3 MB/s | 10 MB 00:01 2024-09-17T03:11:50.733 INFO:teuthology.orchestra.run.smithi087.stdout:(19/157): python3-ceph-argparse-19.3.0-4989.gce 905 kB/s | 45 kB 00:00 2024-09-17T03:11:50.749 INFO:teuthology.orchestra.run.smithi149.stdout:(6/157): ceph-mon-19.3.0-4989.gce1b342b.el9.x86 15 MB/s | 5.0 MB 00:00 2024-09-17T03:11:50.850 INFO:teuthology.orchestra.run.smithi087.stdout:(20/157): librgw2-19.3.0-4989.gce1b342b.el9.x86 14 MB/s | 5.3 MB 00:00 2024-09-17T03:11:50.876 INFO:teuthology.orchestra.run.smithi087.stdout:(21/157): python3-ceph-common-19.3.0-4989.gce1b 1.0 MB/s | 143 kB 00:00 2024-09-17T03:11:50.918 INFO:teuthology.orchestra.run.smithi087.stdout:(22/157): python3-cephfs-19.3.0-4989.gce1b342b. 2.4 MB/s | 162 kB 00:00 2024-09-17T03:11:50.924 INFO:teuthology.orchestra.run.smithi149.stdout:(7/157): ceph-base-19.3.0-4989.gce1b342b.el9.x8 2.8 MB/s | 5.5 MB 00:01 2024-09-17T03:11:50.942 INFO:teuthology.orchestra.run.smithi087.stdout:(23/157): python3-rados-19.3.0-4989.gce1b342b.e 4.7 MB/s | 321 kB 00:00 2024-09-17T03:11:50.993 INFO:teuthology.orchestra.run.smithi087.stdout:(24/157): python3-rbd-19.3.0-4989.gce1b342b.el9 4.0 MB/s | 305 kB 00:00 2024-09-17T03:11:51.021 INFO:teuthology.orchestra.run.smithi087.stdout:(25/157): python3-rgw-19.3.0-4989.gce1b342b.el9 1.3 MB/s | 99 kB 00:00 2024-09-17T03:11:51.069 INFO:teuthology.orchestra.run.smithi087.stdout:(26/157): rbd-fuse-19.3.0-4989.gce1b342b.el9.x8 1.1 MB/s | 86 kB 00:00 2024-09-17T03:11:51.152 INFO:teuthology.orchestra.run.smithi087.stdout:(27/157): rbd-nbd-19.3.0-4989.gce1b342b.el9.x86 2.0 MB/s | 170 kB 00:00 2024-09-17T03:11:51.519 INFO:teuthology.orchestra.run.smithi087.stdout:(28/157): ceph-test-19.3.0-4989.gce1b342b.el9.x 18 MB/s | 40 MB 00:02 2024-09-17T03:11:51.533 INFO:teuthology.orchestra.run.smithi149.stdout:(8/157): ceph-osd-19.3.0-4989.gce1b342b.el9.x86 23 MB/s | 18 MB 00:00 2024-09-17T03:11:51.545 INFO:teuthology.orchestra.run.smithi087.stdout:(29/157): ceph-grafana-dashboards-19.3.0-4989.g 87 kB/s | 34 kB 00:00 2024-09-17T03:11:51.570 INFO:teuthology.orchestra.run.smithi087.stdout:(30/157): ceph-mgr-cephadm-19.3.0-4989.gce1b342 3.0 MB/s | 154 kB 00:00 2024-09-17T03:11:51.575 INFO:teuthology.orchestra.run.smithi149.stdout:(9/157): ceph-mgr-19.3.0-4989.gce1b342b.el9.x86 1.1 MB/s | 1.5 MB 00:01 2024-09-17T03:11:51.601 INFO:teuthology.orchestra.run.smithi149.stdout:(10/157): ceph-selinux-19.3.0-4989.gce1b342b.el 376 kB/s | 25 kB 00:00 2024-09-17T03:11:51.628 INFO:teuthology.orchestra.run.smithi087.stdout:(31/157): rbd-mirror-19.3.0-4989.gce1b342b.el9. 5.1 MB/s | 3.1 MB 00:00 2024-09-17T03:11:51.651 INFO:teuthology.orchestra.run.smithi149.stdout:(11/157): cephfs-mirror-19.3.0-4989.gce1b342b.e 4.4 MB/s | 225 kB 00:00 2024-09-17T03:11:51.712 INFO:teuthology.orchestra.run.smithi087.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4989.gce 3.4 MB/s | 291 kB 00:00 2024-09-17T03:11:51.760 INFO:teuthology.orchestra.run.smithi149.stdout:(12/157): cephfs-shell-19.3.0-4989.gce1b342b.el 249 kB/s | 27 kB 00:00 2024-09-17T03:11:51.787 INFO:teuthology.orchestra.run.smithi087.stdout:(33/157): ceph-mgr-rook-19.3.0-4989.gce1b342b.e 661 kB/s | 49 kB 00:00 2024-09-17T03:11:51.835 INFO:teuthology.orchestra.run.smithi149.stdout:(13/157): libcephfs-devel-19.3.0-4989.gce1b342b 423 kB/s | 32 kB 00:00 2024-09-17T03:11:51.862 INFO:teuthology.orchestra.run.smithi087.stdout:(34/157): ceph-prometheus-alerts-19.3.0-4989.gc 220 kB/s | 16 kB 00:00 2024-09-17T03:11:51.935 INFO:teuthology.orchestra.run.smithi149.stdout:(14/157): libcephfs2-19.3.0-4989.gce1b342b.el9. 7.5 MB/s | 766 kB 00:00 2024-09-17T03:11:51.986 INFO:teuthology.orchestra.run.smithi149.stdout:(15/157): libcephsqlite-19.3.0-4989.gce1b342b.e 3.1 MB/s | 159 kB 00:00 2024-09-17T03:11:52.004 INFO:teuthology.orchestra.run.smithi087.stdout:(35/157): ceph-mgr-diskprediction-local-19.3.0- 17 MB/s | 7.4 MB 00:00 2024-09-17T03:11:52.036 INFO:teuthology.orchestra.run.smithi149.stdout:(16/157): librados-devel-19.3.0-4989.gce1b342b. 2.5 MB/s | 127 kB 00:00 2024-09-17T03:11:52.038 INFO:teuthology.orchestra.run.smithi087.stdout:(36/157): ceph-volume-19.3.0-4989.gce1b342b.el9 1.6 MB/s | 285 kB 00:00 2024-09-17T03:11:52.086 INFO:teuthology.orchestra.run.smithi149.stdout:(17/157): libradosstriper1-19.3.0-4989.gce1b342 9.9 MB/s | 507 kB 00:00 2024-09-17T03:11:52.105 INFO:teuthology.orchestra.run.smithi087.stdout:(37/157): cephadm-19.3.0-4989.gce1b342b.el9.noa 7.7 MB/s | 784 kB 00:00 2024-09-17T03:11:52.130 INFO:teuthology.orchestra.run.smithi087.stdout:(38/157): cephfs-top-19.3.0-4989.gce1b342b.el9. 199 kB/s | 18 kB 00:00 2024-09-17T03:11:52.286 INFO:teuthology.orchestra.run.smithi149.stdout:(18/157): librgw2-19.3.0-4989.gce1b342b.el9.x86 26 MB/s | 5.3 MB 00:00 2024-09-17T03:11:52.289 INFO:teuthology.orchestra.run.smithi087.stdout:(39/157): ceph-mgr-dashboard-19.3.0-4989.gce1b3 19 MB/s | 14 MB 00:00 2024-09-17T03:11:52.336 INFO:teuthology.orchestra.run.smithi149.stdout:(19/157): python3-ceph-argparse-19.3.0-4989.gce 908 kB/s | 45 kB 00:00 2024-09-17T03:11:52.378 INFO:teuthology.orchestra.run.smithi149.stdout:(20/157): python3-ceph-common-19.3.0-4989.gce1b 3.4 MB/s | 143 kB 00:00 2024-09-17T03:11:52.381 INFO:teuthology.orchestra.run.smithi087.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 290 kB/s | 80 kB 00:00 2024-09-17T03:11:52.406 INFO:teuthology.orchestra.run.smithi087.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.6 MB/s | 191 kB 00:00 2024-09-17T03:11:52.429 INFO:teuthology.orchestra.run.smithi149.stdout:(21/157): python3-cephfs-19.3.0-4989.gce1b342b. 3.2 MB/s | 162 kB 00:00 2024-09-17T03:11:52.465 INFO:teuthology.orchestra.run.smithi087.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.3 MB/s | 801 kB 00:00 2024-09-17T03:11:52.479 INFO:teuthology.orchestra.run.smithi149.stdout:(22/157): python3-rados-19.3.0-4989.gce1b342b.e 6.3 MB/s | 321 kB 00:00 2024-09-17T03:11:52.490 INFO:teuthology.orchestra.run.smithi087.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-09-17T03:11:52.515 INFO:teuthology.orchestra.run.smithi087.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-17T03:11:52.529 INFO:teuthology.orchestra.run.smithi149.stdout:(23/157): python3-rbd-19.3.0-4989.gce1b342b.el9 6.0 MB/s | 305 kB 00:00 2024-09-17T03:11:52.549 INFO:teuthology.orchestra.run.smithi087.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-09-17T03:11:52.571 INFO:teuthology.orchestra.run.smithi149.stdout:(24/157): python3-rgw-19.3.0-4989.gce1b342b.el9 2.3 MB/s | 99 kB 00:00 2024-09-17T03:11:52.574 INFO:teuthology.orchestra.run.smithi087.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-17T03:11:52.600 INFO:teuthology.orchestra.run.smithi087.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-17T03:11:52.622 INFO:teuthology.orchestra.run.smithi149.stdout:(25/157): rbd-fuse-19.3.0-4989.gce1b342b.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-17T03:11:52.641 INFO:teuthology.orchestra.run.smithi087.stdout:(48/157): python3-cryptography-36.0.1-4.el9.x86 5.2 MB/s | 1.2 MB 00:00 2024-09-17T03:11:52.764 INFO:teuthology.orchestra.run.smithi149.stdout:(26/157): rbd-mirror-19.3.0-4989.gce1b342b.el9. 22 MB/s | 3.1 MB 00:00 2024-09-17T03:11:52.814 INFO:teuthology.orchestra.run.smithi149.stdout:(27/157): rbd-nbd-19.3.0-4989.gce1b342b.el9.x86 3.3 MB/s | 170 kB 00:00 2024-09-17T03:11:52.856 INFO:teuthology.orchestra.run.smithi149.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4989.g 826 kB/s | 34 kB 00:00 2024-09-17T03:11:52.906 INFO:teuthology.orchestra.run.smithi149.stdout:(29/157): ceph-mgr-cephadm-19.3.0-4989.gce1b342 3.0 MB/s | 154 kB 00:00 2024-09-17T03:11:53.048 INFO:teuthology.orchestra.run.smithi149.stdout:(30/157): ceph-radosgw-19.3.0-4989.gce1b342b.el 4.9 MB/s | 10 MB 00:02 2024-09-17T03:11:53.108 INFO:teuthology.orchestra.run.smithi087.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 229 kB/s | 107 kB 00:00 2024-09-17T03:11:53.217 INFO:teuthology.orchestra.run.smithi087.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 200 kB/s | 22 kB 00:00 2024-09-17T03:11:53.400 INFO:teuthology.orchestra.run.smithi087.stdout:(51/157): automake-1.16.2-8.el9.noarch.rpm 834 kB/s | 689 kB 00:00 2024-09-17T03:11:53.426 INFO:teuthology.orchestra.run.smithi087.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-09-17T03:11:53.432 INFO:teuthology.orchestra.run.smithi149.stdout:(31/157): ceph-mgr-dashboard-19.3.0-4989.gce1b3 27 MB/s | 14 MB 00:00 2024-09-17T03:11:53.484 INFO:teuthology.orchestra.run.smithi149.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4989.gce 5.7 MB/s | 291 kB 00:00 2024-09-17T03:11:53.509 INFO:teuthology.orchestra.run.smithi087.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-09-17T03:11:53.524 INFO:teuthology.orchestra.run.smithi149.stdout:(33/157): ceph-mgr-rook-19.3.0-4989.gce1b342b.e 1.2 MB/s | 49 kB 00:00 2024-09-17T03:11:53.558 INFO:teuthology.orchestra.run.smithi149.stdout:(34/157): ceph-prometheus-alerts-19.3.0-4989.gc 497 kB/s | 16 kB 00:00 2024-09-17T03:11:53.601 INFO:teuthology.orchestra.run.smithi087.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 817 kB/s | 313 kB 00:00 2024-09-17T03:11:53.608 INFO:teuthology.orchestra.run.smithi149.stdout:(35/157): ceph-volume-19.3.0-4989.gce1b342b.el9 5.6 MB/s | 285 kB 00:00 2024-09-17T03:11:53.626 INFO:teuthology.orchestra.run.smithi087.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 156 kB/s | 18 kB 00:00 2024-09-17T03:11:53.667 INFO:teuthology.orchestra.run.smithi149.stdout:(36/157): cephadm-19.3.0-4989.gce1b342b.el9.noa 13 MB/s | 784 kB 00:00 2024-09-17T03:11:53.708 INFO:teuthology.orchestra.run.smithi149.stdout:(37/157): cephfs-top-19.3.0-4989.gce1b342b.el9. 440 kB/s | 18 kB 00:00 2024-09-17T03:11:53.710 INFO:teuthology.orchestra.run.smithi087.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 102 kB/s | 11 kB 00:00 2024-09-17T03:11:53.744 INFO:teuthology.orchestra.run.smithi087.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.4 MB/s | 171 kB 00:00 2024-09-17T03:11:53.844 INFO:teuthology.orchestra.run.smithi087.stdout:(58/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.2 MB/s | 3.0 MB 00:00 2024-09-17T03:11:53.851 INFO:teuthology.orchestra.run.smithi149.stdout:(38/157): fuse-2.9.9-16.el9.x86_64.rpm 562 kB/s | 80 kB 00:00 2024-09-17T03:11:53.869 INFO:teuthology.orchestra.run.smithi087.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 361 kB/s | 45 kB 00:00 2024-09-17T03:11:53.896 INFO:teuthology.orchestra.run.smithi087.stdout:(60/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 870 kB/s | 160 kB 00:00 2024-09-17T03:11:53.962 INFO:teuthology.orchestra.run.smithi087.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-17T03:11:54.003 INFO:teuthology.orchestra.run.smithi087.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 225 kB/s | 24 kB 00:00 2024-09-17T03:11:54.070 INFO:teuthology.orchestra.run.smithi087.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-09-17T03:11:54.121 INFO:teuthology.orchestra.run.smithi087.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-17T03:11:54.134 INFO:teuthology.orchestra.run.smithi149.stdout:(39/157): ceph-mgr-diskprediction-local-19.3.0- 6.8 MB/s | 7.4 MB 00:01 2024-09-17T03:11:54.154 INFO:teuthology.orchestra.run.smithi087.stdout:(65/157): libtool-2.4.6-46.el9.x86_64.rpm 2.0 MB/s | 581 kB 00:00 2024-09-17T03:11:54.168 INFO:teuthology.orchestra.run.smithi149.stdout:(40/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.5 MB/s | 801 kB 00:00 2024-09-17T03:11:54.179 INFO:teuthology.orchestra.run.smithi087.stdout:(66/157): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-09-17T03:11:54.238 INFO:teuthology.orchestra.run.smithi087.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 328 kB/s | 38 kB 00:00 2024-09-17T03:11:54.277 INFO:teuthology.orchestra.run.smithi149.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-09-17T03:11:54.304 INFO:teuthology.orchestra.run.smithi149.stdout:(42/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-17T03:11:54.352 INFO:teuthology.orchestra.run.smithi149.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-17T03:11:54.527 INFO:teuthology.orchestra.run.smithi149.stdout:(44/157): python3-pycparser-2.20-6.el9.noarch.r 773 kB/s | 135 kB 00:00 2024-09-17T03:11:54.555 INFO:teuthology.orchestra.run.smithi087.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 5.0 MB/s | 1.6 MB 00:00 2024-09-17T03:11:54.611 INFO:teuthology.orchestra.run.smithi149.stdout:(45/157): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-17T03:11:54.695 INFO:teuthology.orchestra.run.smithi149.stdout:(46/157): python3-cryptography-36.0.1-4.el9.x86 2.9 MB/s | 1.2 MB 00:00 2024-09-17T03:11:54.720 INFO:teuthology.orchestra.run.smithi149.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.9 MB/s | 215 kB 00:00 2024-09-17T03:11:54.855 INFO:teuthology.orchestra.run.smithi087.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 3.0 MB/s | 910 kB 00:00 2024-09-17T03:11:54.895 INFO:teuthology.orchestra.run.smithi149.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-09-17T03:11:54.957 INFO:teuthology.orchestra.run.smithi149.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-09-17T03:11:54.979 INFO:teuthology.orchestra.run.smithi149.stdout:(50/157): bison-3.7.4-5.el9.x86_64.rpm 3.6 MB/s | 943 kB 00:00 2024-09-17T03:11:55.004 INFO:teuthology.orchestra.run.smithi149.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 369 kB/s | 22 kB 00:00 2024-09-17T03:11:55.022 INFO:teuthology.orchestra.run.smithi087.stdout:(70/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 5.3 MB/s | 4.4 MB 00:00 2024-09-17T03:11:55.047 INFO:teuthology.orchestra.run.smithi087.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 717 kB/s | 137 kB 00:00 2024-09-17T03:11:55.054 INFO:teuthology.orchestra.run.smithi149.stdout:(52/157): flexiblas-3.0.4-8.el9.x86_64.rpm 656 kB/s | 33 kB 00:00 2024-09-17T03:11:55.105 INFO:teuthology.orchestra.run.smithi149.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-09-17T03:11:55.155 INFO:teuthology.orchestra.run.smithi149.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-09-17T03:11:55.206 INFO:teuthology.orchestra.run.smithi149.stdout:(55/157): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-17T03:11:55.297 INFO:teuthology.orchestra.run.smithi149.stdout:(56/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.8 MB/s | 171 kB 00:00 2024-09-17T03:11:55.348 INFO:teuthology.orchestra.run.smithi087.stdout:(72/157): protobuf-3.14.0-13.el9.x86_64.rpm 3.4 MB/s | 1.0 MB 00:00 2024-09-17T03:11:55.381 INFO:teuthology.orchestra.run.smithi149.stdout:(57/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-17T03:11:55.431 INFO:teuthology.orchestra.run.smithi149.stdout:(58/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 908 kB/s | 45 kB 00:00 2024-09-17T03:11:55.456 INFO:teuthology.orchestra.run.smithi087.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 808 kB/s | 87 kB 00:00 2024-09-17T03:11:55.598 INFO:teuthology.orchestra.run.smithi087.stdout:(74/157): openblas-openmp-0.3.26-2.el9.x86_64.r 3.4 MB/s | 4.9 MB 00:01 2024-09-17T03:11:55.607 INFO:teuthology.orchestra.run.smithi149.stdout:(59/157): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 662 kB 00:00 2024-09-17T03:11:55.698 INFO:teuthology.orchestra.run.smithi087.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 614 kB/s | 61 kB 00:00 2024-09-17T03:11:55.774 INFO:teuthology.orchestra.run.smithi149.stdout:(60/157): libtool-2.4.6-46.el9.x86_64.rpm 3.4 MB/s | 581 kB 00:00 2024-09-17T03:11:55.832 INFO:teuthology.orchestra.run.smithi149.stdout:(61/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.8 MB/s | 3.0 MB 00:00 2024-09-17T03:11:55.849 INFO:teuthology.orchestra.run.smithi149.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 326 kB/s | 24 kB 00:00 2024-09-17T03:11:55.883 INFO:teuthology.orchestra.run.smithi149.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-17T03:11:55.941 INFO:teuthology.orchestra.run.smithi149.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-17T03:11:55.967 INFO:teuthology.orchestra.run.smithi149.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-17T03:11:55.992 INFO:teuthology.orchestra.run.smithi149.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 766 kB/s | 38 kB 00:00 2024-09-17T03:11:56.082 INFO:teuthology.orchestra.run.smithi087.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 638 kB/s | 245 kB 00:00 2024-09-17T03:11:56.482 INFO:teuthology.orchestra.run.smithi087.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 620 kB/s | 248 kB 00:00 2024-09-17T03:11:56.591 INFO:teuthology.orchestra.run.smithi087.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 237 kB/s | 26 kB 00:00 2024-09-17T03:11:56.733 INFO:teuthology.orchestra.run.smithi087.stdout:(79/157): python3-babel-2.9.1-2.el9.noarch.rpm 4.7 MB/s | 6.0 MB 00:01 2024-09-17T03:11:56.758 INFO:teuthology.orchestra.run.smithi087.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 115 kB/s | 19 kB 00:00 2024-09-17T03:11:56.826 INFO:teuthology.orchestra.run.smithi149.stdout:(67/157): openblas-openmp-0.3.26-2.el9.x86_64.r 5.8 MB/s | 4.9 MB 00:00 2024-09-17T03:11:56.850 INFO:teuthology.orchestra.run.smithi087.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-17T03:11:56.875 INFO:teuthology.orchestra.run.smithi087.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 297 kB/s | 35 kB 00:00 2024-09-17T03:11:56.968 INFO:teuthology.orchestra.run.smithi149.stdout:(68/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 4.5 MB/s | 4.4 MB 00:00 2024-09-17T03:11:57.135 INFO:teuthology.orchestra.run.smithi149.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 5.1 MB/s | 1.6 MB 00:00 2024-09-17T03:11:57.177 INFO:teuthology.orchestra.run.smithi149.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 4.3 MB/s | 910 kB 00:00 2024-09-17T03:11:57.202 INFO:teuthology.orchestra.run.smithi149.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-17T03:11:57.259 INFO:teuthology.orchestra.run.smithi087.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 654 kB/s | 250 kB 00:00 2024-09-17T03:11:57.344 INFO:teuthology.orchestra.run.smithi149.stdout:(72/157): protobuf-3.14.0-13.el9.x86_64.rpm 7.1 MB/s | 1.0 MB 00:00 2024-09-17T03:11:57.397 INFO:teuthology.orchestra.run.smithi149.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-17T03:11:57.634 INFO:teuthology.orchestra.run.smithi087.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 593 kB/s | 222 kB 00:00 2024-09-17T03:11:57.835 INFO:teuthology.orchestra.run.smithi087.stdout:(85/157): python3-packaging-20.9-5.el9.noarch.r 386 kB/s | 77 kB 00:00 2024-09-17T03:11:57.901 INFO:teuthology.orchestra.run.smithi087.stdout:(86/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.8 MB/s | 5.1 MB 00:01 2024-09-17T03:11:57.943 INFO:teuthology.orchestra.run.smithi087.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 389 kB/s | 42 kB 00:00 2024-09-17T03:11:57.944 INFO:teuthology.orchestra.run.smithi149.stdout:(74/157): postgresql-server-13.16-1.el9.x86_64. 7.5 MB/s | 5.8 MB 00:00 2024-09-17T03:11:58.019 INFO:teuthology.orchestra.run.smithi087.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 269 kB 00:00 2024-09-17T03:11:58.028 INFO:teuthology.orchestra.run.smithi149.stdout:(75/157): python3-babel-2.9.1-2.el9.noarch.rpm 9.4 MB/s | 6.0 MB 00:00 2024-09-17T03:11:58.053 INFO:teuthology.orchestra.run.smithi149.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 565 kB/s | 61 kB 00:00 2024-09-17T03:11:58.079 INFO:teuthology.orchestra.run.smithi149.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-17T03:11:58.112 INFO:teuthology.orchestra.run.smithi149.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-17T03:11:58.138 INFO:teuthology.orchestra.run.smithi149.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 438 kB/s | 26 kB 00:00 2024-09-17T03:11:58.143 INFO:teuthology.orchestra.run.smithi087.stdout:(89/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-09-17T03:11:58.163 INFO:teuthology.orchestra.run.smithi149.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-17T03:11:58.188 INFO:teuthology.orchestra.run.smithi149.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-17T03:11:58.217 INFO:teuthology.orchestra.run.smithi149.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-09-17T03:11:58.269 INFO:teuthology.orchestra.run.smithi087.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 488 kB/s | 159 kB 00:00 2024-09-17T03:11:58.272 INFO:teuthology.orchestra.run.smithi149.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-17T03:11:58.294 INFO:teuthology.orchestra.run.smithi087.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 339 kB/s | 51 kB 00:00 2024-09-17T03:11:58.386 INFO:teuthology.orchestra.run.smithi087.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 459 kB/s | 54 kB 00:00 2024-09-17T03:11:58.503 INFO:teuthology.orchestra.run.smithi087.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 357 kB/s | 42 kB 00:00 2024-09-17T03:11:58.612 INFO:teuthology.orchestra.run.smithi087.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 407 kB/s | 44 kB 00:00 2024-09-17T03:11:58.631 INFO:teuthology.orchestra.run.smithi149.stdout:(84/157): ceph-test-19.3.0-4989.gce1b342b.el9.x 5.7 MB/s | 40 MB 00:07 2024-09-17T03:11:58.656 INFO:teuthology.orchestra.run.smithi149.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 579 kB/s | 222 kB 00:00 2024-09-17T03:11:58.706 INFO:teuthology.orchestra.run.smithi149.stdout:(86/157): python3-prettytable-0.7.2-27.el9.noar 846 kB/s | 42 kB 00:00 2024-09-17T03:11:58.773 INFO:teuthology.orchestra.run.smithi149.stdout:(87/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 8.7 MB/s | 5.1 MB 00:00 2024-09-17T03:11:58.798 INFO:teuthology.orchestra.run.smithi149.stdout:(88/157): python3-packaging-20.9-5.el9.noarch.r 462 kB/s | 77 kB 00:00 2024-09-17T03:11:58.824 INFO:teuthology.orchestra.run.smithi149.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-09-17T03:11:58.849 INFO:teuthology.orchestra.run.smithi149.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-09-17T03:11:58.874 INFO:teuthology.orchestra.run.smithi149.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-17T03:11:58.908 INFO:teuthology.orchestra.run.smithi149.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 915 kB/s | 54 kB 00:00 2024-09-17T03:11:58.933 INFO:teuthology.orchestra.run.smithi149.stdout:(93/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.0 MB/s | 279 kB 00:00 2024-09-17T03:11:58.967 INFO:teuthology.orchestra.run.smithi149.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 713 kB/s | 42 kB 00:00 2024-09-17T03:11:58.987 INFO:teuthology.orchestra.run.smithi087.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 588 kB/s | 221 kB 00:00 2024-09-17T03:11:59.000 INFO:teuthology.orchestra.run.smithi149.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 657 kB/s | 44 kB 00:00 2024-09-17T03:11:59.026 INFO:teuthology.orchestra.run.smithi149.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-17T03:11:59.052 INFO:teuthology.orchestra.run.smithi149.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-09-17T03:11:59.076 INFO:teuthology.orchestra.run.smithi149.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-17T03:11:59.104 INFO:teuthology.orchestra.run.smithi087.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 308 kB/s | 36 kB 00:00 2024-09-17T03:11:59.127 INFO:teuthology.orchestra.run.smithi149.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-17T03:11:59.177 INFO:teuthology.orchestra.run.smithi149.stdout:(100/157): socat-1.7.4.1-6.el9.x86_64.rpm 2.4 MB/s | 304 kB 00:00 2024-09-17T03:11:59.202 INFO:teuthology.orchestra.run.smithi149.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.2 MB/s | 96 kB 00:00 2024-09-17T03:11:59.227 INFO:teuthology.orchestra.run.smithi149.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-17T03:11:59.238 INFO:teuthology.orchestra.run.smithi087.stdout:(97/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 355 kB/s | 47 kB 00:00 2024-09-17T03:11:59.253 INFO:teuthology.orchestra.run.smithi149.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-09-17T03:11:59.495 INFO:teuthology.orchestra.run.smithi149.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 83 kB/s | 22 kB 00:00 2024-09-17T03:11:59.578 INFO:teuthology.orchestra.run.smithi149.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 6.4 MB/s | 548 kB 00:00 2024-09-17T03:11:59.620 INFO:teuthology.orchestra.run.smithi149.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 7.2 MB/s | 308 kB 00:00 2024-09-17T03:11:59.637 INFO:teuthology.orchestra.run.smithi149.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 1.2 MB/s | 19 kB 00:00 2024-09-17T03:11:59.730 INFO:teuthology.orchestra.run.smithi087.stdout:(98/157): socat-1.7.4.1-6.el9.x86_64.rpm 618 kB/s | 304 kB 00:00 2024-09-17T03:11:59.796 INFO:teuthology.orchestra.run.smithi149.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-09-17T03:11:59.821 INFO:teuthology.orchestra.run.smithi149.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-17T03:11:59.846 INFO:teuthology.orchestra.run.smithi149.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-17T03:11:59.847 INFO:teuthology.orchestra.run.smithi087.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 495 kB/s | 58 kB 00:00 2024-09-17T03:11:59.872 INFO:teuthology.orchestra.run.smithi149.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-17T03:11:59.897 INFO:teuthology.orchestra.run.smithi149.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-17T03:11:59.923 INFO:teuthology.orchestra.run.smithi149.stdout:(113/157): protobuf-compiler-3.14.0-13.el9.x86_ 1.3 MB/s | 863 kB 00:00 2024-09-17T03:11:59.964 INFO:teuthology.orchestra.run.smithi149.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-17T03:11:59.989 INFO:teuthology.orchestra.run.smithi149.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.8 MB/s | 43 kB 00:00 2024-09-17T03:12:00.015 INFO:teuthology.orchestra.run.smithi149.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-17T03:12:00.040 INFO:teuthology.orchestra.run.smithi149.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 570 kB/s | 14 kB 00:00 2024-09-17T03:12:00.047 INFO:teuthology.orchestra.run.smithi087.stdout:(100/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 479 kB/s | 96 kB 00:00 2024-09-17T03:12:00.065 INFO:teuthology.orchestra.run.smithi149.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-17T03:12:00.091 INFO:teuthology.orchestra.run.smithi149.stdout:(119/157): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-09-17T03:12:00.133 INFO:teuthology.orchestra.run.smithi149.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-17T03:12:00.159 INFO:teuthology.orchestra.run.smithi149.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-09-17T03:12:00.192 INFO:teuthology.orchestra.run.smithi149.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-17T03:12:00.217 INFO:teuthology.orchestra.run.smithi149.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-09-17T03:12:00.242 INFO:teuthology.orchestra.run.smithi149.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-09-17T03:12:00.247 INFO:teuthology.orchestra.run.smithi087.stdout:(101/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 318 kB/s | 64 kB 00:00 2024-09-17T03:12:00.267 INFO:teuthology.orchestra.run.smithi149.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 710 kB/s | 18 kB 00:00 2024-09-17T03:12:00.293 INFO:teuthology.orchestra.run.smithi149.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 924 kB/s | 23 kB 00:00 2024-09-17T03:12:00.318 INFO:teuthology.orchestra.run.smithi149.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 785 kB/s | 19 kB 00:00 2024-09-17T03:12:00.344 INFO:teuthology.orchestra.run.smithi149.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 787 kB/s | 20 kB 00:00 2024-09-17T03:12:00.364 INFO:teuthology.orchestra.run.smithi087.stdout:(102/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 384 kB/s | 45 kB 00:00 2024-09-17T03:12:00.464 INFO:teuthology.orchestra.run.smithi087.stdout:(103/157): lua-devel-5.4.4-4.el9.x86_64.rpm 224 kB/s | 22 kB 00:00 2024-09-17T03:12:00.511 INFO:teuthology.orchestra.run.smithi149.stdout:(129/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 9.9 MB/s | 16 MB 00:01 2024-09-17T03:12:00.561 INFO:teuthology.orchestra.run.smithi149.stdout:(130/157): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-09-17T03:12:00.603 INFO:teuthology.orchestra.run.smithi149.stdout:(131/157): python3-grpcio-1.46.7-10.el9.x86_64. 4.6 MB/s | 2.0 MB 00:00 2024-09-17T03:12:00.628 INFO:teuthology.orchestra.run.smithi149.stdout:(132/157): python3-logutils-0.3.5-21.el9.noarch 396 kB/s | 46 kB 00:00 2024-09-17T03:12:00.631 INFO:teuthology.orchestra.run.smithi087.stdout:(104/157): protobuf-compiler-3.14.0-13.el9.x86_ 5.1 MB/s | 863 kB 00:00 2024-09-17T03:12:00.653 INFO:teuthology.orchestra.run.smithi149.stdout:(133/157): python3-more-itertools-8.12.0-2.el9. 875 kB/s | 79 kB 00:00 2024-09-17T03:12:00.679 INFO:teuthology.orchestra.run.smithi149.stdout:(134/157): python3-natsort-7.1.1-5.el9.noarch.r 765 kB/s | 58 kB 00:00 2024-09-17T03:12:00.704 INFO:teuthology.orchestra.run.smithi149.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-09-17T03:12:00.706 INFO:teuthology.orchestra.run.smithi087.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 7.1 MB/s | 548 kB 00:00 2024-09-17T03:12:00.729 INFO:teuthology.orchestra.run.smithi149.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-17T03:12:00.740 INFO:teuthology.orchestra.run.smithi087.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-17T03:12:00.754 INFO:teuthology.orchestra.run.smithi149.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-17T03:12:00.765 INFO:teuthology.orchestra.run.smithi087.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 781 kB/s | 19 kB 00:00 2024-09-17T03:12:00.781 INFO:teuthology.orchestra.run.smithi149.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-17T03:12:00.805 INFO:teuthology.orchestra.run.smithi149.stdout:(139/157): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-09-17T03:12:00.830 INFO:teuthology.orchestra.run.smithi149.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-17T03:12:00.855 INFO:teuthology.orchestra.run.smithi149.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 716 kB/s | 36 kB 00:00 2024-09-17T03:12:00.881 INFO:teuthology.orchestra.run.smithi149.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-17T03:12:00.906 INFO:teuthology.orchestra.run.smithi149.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 988 kB/s | 74 kB 00:00 2024-09-17T03:12:00.924 INFO:teuthology.orchestra.run.smithi087.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-09-17T03:12:00.931 INFO:teuthology.orchestra.run.smithi149.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-17T03:12:00.949 INFO:teuthology.orchestra.run.smithi087.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-17T03:12:00.957 INFO:teuthology.orchestra.run.smithi149.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-17T03:12:00.974 INFO:teuthology.orchestra.run.smithi149.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 477 kB/s | 20 kB 00:00 2024-09-17T03:12:00.974 INFO:teuthology.orchestra.run.smithi087.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-17T03:12:00.999 INFO:teuthology.orchestra.run.smithi149.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-17T03:12:01.000 INFO:teuthology.orchestra.run.smithi087.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-17T03:12:01.024 INFO:teuthology.orchestra.run.smithi149.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-09-17T03:12:01.033 INFO:teuthology.orchestra.run.smithi087.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-09-17T03:12:01.083 INFO:teuthology.orchestra.run.smithi149.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-17T03:12:01.192 INFO:teuthology.orchestra.run.smithi087.stdout:(113/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.6 MB/s | 16 MB 00:02 2024-09-17T03:12:01.242 INFO:teuthology.orchestra.run.smithi087.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.9 MB/s | 839 kB 00:00 2024-09-17T03:12:01.276 INFO:teuthology.orchestra.run.smithi087.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.4 MB/s | 43 kB 00:00 2024-09-17T03:12:01.341 INFO:teuthology.orchestra.run.smithi149.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 9.0 MB/s | 2.3 MB 00:00 2024-09-17T03:12:01.409 INFO:teuthology.orchestra.run.smithi087.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 241 kB/s | 32 kB 00:00 2024-09-17T03:12:01.435 INFO:teuthology.orchestra.run.smithi087.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 564 kB/s | 14 kB 00:00 2024-09-17T03:12:01.469 INFO:teuthology.orchestra.run.smithi087.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 5.1 MB/s | 172 kB 00:00 2024-09-17T03:12:01.493 INFO:teuthology.orchestra.run.smithi149.stdout:(151/157): NetworkManager-libnm-1.51.0-1.el9.x8 12 MB/s | 1.8 MB 00:00 2024-09-17T03:12:01.494 INFO:teuthology.orchestra.run.smithi087.stdout:(119/157): python3-asyncssh-2.13.2-5.el9.noarch 1.8 MB/s | 548 kB 00:00 2024-09-17T03:12:01.527 INFO:teuthology.orchestra.run.smithi087.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-17T03:12:01.550 INFO:teuthology.orchestra.run.smithi149.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 699 kB/s | 41 kB 00:00 2024-09-17T03:12:01.552 INFO:teuthology.orchestra.run.smithi087.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-17T03:12:01.578 INFO:teuthology.orchestra.run.smithi087.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-17T03:12:01.609 INFO:teuthology.orchestra.run.smithi149.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 4.2 MB/s | 249 kB 00:00 2024-09-17T03:12:01.612 INFO:teuthology.orchestra.run.smithi087.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-17T03:12:01.637 INFO:teuthology.orchestra.run.smithi087.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 433 kB/s | 11 kB 00:00 2024-09-17T03:12:01.670 INFO:teuthology.orchestra.run.smithi087.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 528 kB/s | 18 kB 00:00 2024-09-17T03:12:01.696 INFO:teuthology.orchestra.run.smithi087.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 923 kB/s | 23 kB 00:00 2024-09-17T03:12:01.722 INFO:teuthology.orchestra.run.smithi087.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 777 kB/s | 19 kB 00:00 2024-09-17T03:12:01.734 INFO:teuthology.orchestra.run.smithi149.stdout:(154/157): openssl-3.2.2-6.el9.x86_64.rpm 11 MB/s | 1.4 MB 00:00 2024-09-17T03:12:01.746 INFO:teuthology.orchestra.run.smithi087.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 795 kB/s | 20 kB 00:00 2024-09-17T03:12:01.838 INFO:teuthology.orchestra.run.smithi087.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 7.1 MB/s | 2.0 MB 00:00 2024-09-17T03:12:01.864 INFO:teuthology.orchestra.run.smithi087.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-17T03:12:01.889 INFO:teuthology.orchestra.run.smithi087.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-17T03:12:01.893 INFO:teuthology.orchestra.run.smithi149.stdout:(155/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 15 MB/s | 2.4 MB 00:00 2024-09-17T03:12:01.914 INFO:teuthology.orchestra.run.smithi087.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-17T03:12:01.948 INFO:teuthology.orchestra.run.smithi087.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-09-17T03:12:01.973 INFO:teuthology.orchestra.run.smithi087.stdout:(134/157): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2024-09-17T03:12:01.998 INFO:teuthology.orchestra.run.smithi087.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-09-17T03:12:02.024 INFO:teuthology.orchestra.run.smithi087.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-17T03:12:02.049 INFO:teuthology.orchestra.run.smithi087.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-17T03:12:02.074 INFO:teuthology.orchestra.run.smithi087.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-17T03:12:02.100 INFO:teuthology.orchestra.run.smithi087.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-17T03:12:02.105 INFO:teuthology.orchestra.run.smithi149.stdout:(156/157): librados2-19.3.0-4989.gce1b342b.el9. 3.1 MB/s | 3.4 MB 00:01 2024-09-17T03:12:02.125 INFO:teuthology.orchestra.run.smithi087.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-17T03:12:02.150 INFO:teuthology.orchestra.run.smithi087.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-17T03:12:02.152 INFO:teuthology.orchestra.run.smithi149.stdout:(157/157): librbd1-19.3.0-4989.gce1b342b.el9.x8 2.8 MB/s | 3.2 MB 00:01 2024-09-17T03:12:02.158 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-09-17T03:12:02.158 INFO:teuthology.orchestra.run.smithi149.stdout:Total 17 MB/s | 228 MB 00:13 2024-09-17T03:12:02.175 INFO:teuthology.orchestra.run.smithi087.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-17T03:12:02.201 INFO:teuthology.orchestra.run.smithi087.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-17T03:12:02.234 INFO:teuthology.orchestra.run.smithi087.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-17T03:12:02.259 INFO:teuthology.orchestra.run.smithi087.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 805 kB/s | 20 kB 00:00 2024-09-17T03:12:02.285 INFO:teuthology.orchestra.run.smithi087.stdout:(146/157): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-09-17T03:12:02.309 INFO:teuthology.orchestra.run.smithi087.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.8 MB/s | 191 kB 00:00 2024-09-17T03:12:02.359 INFO:teuthology.orchestra.run.smithi087.stdout:(148/157): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-09-17T03:12:02.876 INFO:teuthology.orchestra.run.smithi087.stdout:(149/157): librados2-19.3.0-4989.gce1b342b.el9. 6.0 MB/s | 3.4 MB 00:00 2024-09-17T03:12:03.059 INFO:teuthology.orchestra.run.smithi087.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-09-17T03:12:03.130 INFO:teuthology.orchestra.run.smithi087.stdout:(151/157): NetworkManager-libnm-1.51.0-1.el9.x8 27 MB/s | 1.8 MB 00:00 2024-09-17T03:12:03.185 INFO:teuthology.orchestra.run.smithi087.stdout:(152/157): librbd1-19.3.0-4989.gce1b342b.el9.x8 3.8 MB/s | 3.2 MB 00:00 2024-09-17T03:12:03.202 INFO:teuthology.orchestra.run.smithi087.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 544 kB/s | 41 kB 00:00 2024-09-17T03:12:03.269 INFO:teuthology.orchestra.run.smithi087.stdout:(154/157): openssl-3.2.2-6.el9.x86_64.rpm 20 MB/s | 1.4 MB 00:00 2024-09-17T03:12:03.352 INFO:teuthology.orchestra.run.smithi087.stdout:(155/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 29 MB/s | 2.4 MB 00:00 2024-09-17T03:12:03.386 INFO:teuthology.orchestra.run.smithi087.stdout:(156/157): NetworkManager-tui-1.51.0-1.el9.x86_ 1.2 MB/s | 249 kB 00:00 2024-09-17T03:12:03.395 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-09-17T03:12:03.550 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-09-17T03:12:03.551 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-09-17T03:12:03.986 INFO:teuthology.orchestra.run.smithi087.stdout:(157/157): postgresql-server-13.16-1.el9.x86_64 660 kB/s | 5.8 MB 00:08 2024-09-17T03:12:03.992 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2024-09-17T03:12:03.992 INFO:teuthology.orchestra.run.smithi087.stdout:Total 14 MB/s | 228 MB 00:16 2024-09-17T03:12:05.224 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2024-09-17T03:12:05.378 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2024-09-17T03:12:05.378 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2024-09-17T03:12:05.782 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-09-17T03:12:05.792 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-09-17T03:12:07.622 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2024-09-17T03:12:07.622 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2024-09-17T03:12:12.314 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-09-17T03:12:12.408 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-17T03:12:12.450 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-17T03:12:12.508 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-17T03:12:12.542 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-17T03:12:12.567 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-17T03:12:12.617 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-17T03:12:12.952 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-17T03:12:13.060 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librados2-2:19.3.0-4989.gce1b342b.el9.x86_64 8/165 2024-09-17T03:12:13.187 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:19.3.0-4989.gce1b342b.el9.x86_64 8/165 2024-09-17T03:12:13.220 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs2-2:19.3.0-4989.gce1b342b.el9.x86_64 9/165 2024-09-17T03:12:13.304 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephfs2-2:19.3.0-4989.gce1b342b.el9.x86_64 9/165 2024-09-17T03:12:13.341 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rados-2:19.3.0-4989.gce1b342b.el9.x86_64 10/165 2024-09-17T03:12:13.372 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-17T03:12:13.538 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-17T03:12:13.589 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-17T03:12:13.617 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephsqlite-2:19.3.0-4989.gce1b342b.el9.x86_64 14/165 2024-09-17T03:12:13.731 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4989.gce1b342b.el9.x86_64 14/165 2024-09-17T03:12:13.761 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libradosstriper1-2:19.3.0-4989.gce1b342b.el9.x86 15/165 2024-09-17T03:12:13.826 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4989.gce1b342b.el9.x86 15/165 2024-09-17T03:12:13.852 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-17T03:12:13.878 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-17T03:12:13.926 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2024-09-17T03:12:14.040 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-17T03:12:14.071 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-17T03:12:14.188 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-17T03:12:14.261 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-17T03:12:14.360 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-17T03:12:14.392 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-17T03:12:14.399 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-17T03:12:14.425 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-17T03:12:14.480 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-17T03:12:14.510 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-17T03:12:14.577 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-17T03:12:14.811 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-17T03:12:14.819 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-17T03:12:14.920 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-17T03:12:14.925 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : librados2-2:19.3.0-4989.gce1b342b.el9.x86_64 8/165 2024-09-17T03:12:14.948 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-17T03:12:14.983 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-17T03:12:15.051 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librados2-2:19.3.0-4989.gce1b342b.el9.x86_64 8/165 2024-09-17T03:12:15.057 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-17T03:12:15.076 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcephfs2-2:19.3.0-4989.gce1b342b.el9.x86_64 9/165 2024-09-17T03:12:15.100 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-17T03:12:15.141 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-17T03:12:15.151 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libcephfs2-2:19.3.0-4989.gce1b342b.el9.x86_64 9/165 2024-09-17T03:12:15.184 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rados-2:19.3.0-4989.gce1b342b.el9.x86_64 10/165 2024-09-17T03:12:15.185 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-17T03:12:15.217 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-17T03:12:15.227 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-17T03:12:15.244 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-17T03:12:15.314 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-17T03:12:15.367 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-17T03:12:15.398 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-17T03:12:15.412 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-17T03:12:15.442 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-17T03:12:15.471 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-17T03:12:15.498 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcephsqlite-2:19.3.0-4989.gce1b342b.el9.x86_64 14/165 2024-09-17T03:12:15.520 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-17T03:12:15.552 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-common-2:19.3.0-4989.gce1b342b.el9. 34/165 2024-09-17T03:12:15.587 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-argparse-2:19.3.0-4989.gce1b342b.el 35/165 2024-09-17T03:12:15.616 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cephfs-2:19.3.0-4989.gce1b342b.el9.x86_6 36/165 2024-09-17T03:12:15.624 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4989.gce1b342b.el9.x86_64 14/165 2024-09-17T03:12:15.649 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libradosstriper1-2:19.3.0-4989.gce1b342b.el9.x86 15/165 2024-09-17T03:12:15.685 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-17T03:12:15.704 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4989.gce1b342b.el9.x86 15/165 2024-09-17T03:12:15.725 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-17T03:12:15.730 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-17T03:12:15.745 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-17T03:12:15.805 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-17T03:12:15.894 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-17T03:12:15.935 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-17T03:12:15.991 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-17T03:12:16.091 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-17T03:12:16.223 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-17T03:12:16.334 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-17T03:12:16.416 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-17T03:12:16.519 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-17T03:12:16.626 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-17T03:12:16.655 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-17T03:12:16.657 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-17T03:12:16.692 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-17T03:12:16.730 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librados-devel-2:19.3.0-4989.gce1b342b.el9.x86_6 46/165 2024-09-17T03:12:16.769 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-17T03:12:16.773 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-17T03:12:16.809 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-17T03:12:16.847 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-17T03:12:16.921 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-17T03:12:16.924 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-17T03:12:16.965 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-17T03:12:16.983 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-17T03:12:17.005 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-17T03:12:17.048 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-17T03:12:17.075 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-17T03:12:17.081 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-17T03:12:17.109 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-17T03:12:17.139 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-17T03:12:17.178 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-17T03:12:17.212 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-17T03:12:17.231 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-17T03:12:17.245 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-17T03:12:17.270 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-17T03:12:17.279 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-17T03:12:17.326 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-17T03:12:17.328 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-17T03:12:17.371 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-17T03:12:17.408 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-17T03:12:17.413 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-17T03:12:17.441 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-ceph-common-2:19.3.0-4989.gce1b342b.el9. 34/165 2024-09-17T03:12:17.455 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-17T03:12:17.484 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-ceph-argparse-2:19.3.0-4989.gce1b342b.el 35/165 2024-09-17T03:12:17.513 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cephfs-2:19.3.0-4989.gce1b342b.el9.x86_6 36/165 2024-09-17T03:12:17.583 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-17T03:12:17.592 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-17T03:12:17.627 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-17T03:12:17.632 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-17T03:12:17.674 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-17T03:12:17.702 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-17T03:12:17.776 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-17T03:12:17.808 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-17T03:12:17.833 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-17T03:12:17.887 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-17T03:12:17.895 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-17T03:12:18.073 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-17T03:12:18.224 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-17T03:12:18.322 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-17T03:12:18.426 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-17T03:12:18.448 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-17T03:12:18.493 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-17T03:12:18.517 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-17T03:12:18.521 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-17T03:12:18.546 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-17T03:12:18.554 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-17T03:12:18.589 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-17T03:12:18.593 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-17T03:12:18.625 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-17T03:12:18.634 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librados-devel-2:19.3.0-4989.gce1b342b.el9.x86_6 46/165 2024-09-17T03:12:18.655 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-17T03:12:18.683 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-17T03:12:18.689 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-17T03:12:18.723 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-17T03:12:18.756 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-17T03:12:18.791 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-17T03:12:18.856 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-17T03:12:18.878 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-17T03:12:18.921 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-17T03:12:18.921 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-17T03:12:18.952 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-17T03:12:19.020 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-17T03:12:19.055 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-17T03:12:19.086 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-17T03:12:19.103 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-17T03:12:19.158 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-17T03:12:19.198 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-17T03:12:19.234 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-17T03:12:19.281 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-17T03:12:19.317 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-17T03:12:19.360 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-17T03:12:19.401 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-17T03:12:19.537 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-17T03:12:19.965 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-17T03:12:19.965 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-17T03:12:19.965 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-17T03:12:19.966 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-17T03:12:19.967 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-17T03:12:19.967 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-17T03:12:19.967 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-17T03:12:19.967 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-17T03:12:19.967 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-17T03:12:19.967 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-17T03:12:20.000 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-17T03:12:20.056 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-17T03:12:20.073 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-17T03:12:20.101 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-17T03:12:20.196 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-17T03:12:20.200 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-17T03:12:20.228 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-17T03:12:20.237 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-17T03:12:20.270 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-17T03:12:20.299 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-17T03:12:20.347 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-17T03:12:20.378 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-17T03:12:20.408 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-17T03:12:20.415 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-17T03:12:20.442 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-17T03:12:20.445 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-17T03:12:20.476 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-17T03:12:20.517 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-17T03:12:20.523 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-17T03:12:20.552 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-17T03:12:20.647 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-17T03:12:20.690 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-17T03:12:20.730 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-17T03:12:20.832 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-17T03:12:20.889 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-17T03:12:21.170 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-17T03:12:21.276 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-17T03:12:21.607 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-17T03:12:21.668 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-17T03:12:21.701 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-17T03:12:21.799 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-17T03:12:21.848 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-17T03:12:21.893 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-17T03:12:21.981 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-17T03:12:22.031 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-17T03:12:22.216 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-17T03:12:22.254 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-17T03:12:22.332 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-17T03:12:22.528 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-17T03:12:22.616 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-17T03:12:22.644 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-17T03:12:22.677 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-17T03:12:22.994 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-17T03:12:23.001 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-17T03:12:23.048 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librbd1-2:19.3.0-4989.gce1b342b.el9.x86_64 102/165 2024-09-17T03:12:23.101 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-17T03:12:23.139 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:19.3.0-4989.gce1b342b.el9.x86_64 102/165 2024-09-17T03:12:23.170 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rbd-2:19.3.0-4989.gce1b342b.el9.x86_64 103/165 2024-09-17T03:12:23.206 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-17T03:12:23.703 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-17T03:12:23.735 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librgw2-2:19.3.0-4989.gce1b342b.el9.x86_64 106/165 2024-09-17T03:12:23.806 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librgw2-2:19.3.0-4989.gce1b342b.el9.x86_64 106/165 2024-09-17T03:12:23.836 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rgw-2:19.3.0-4989.gce1b342b.el9.x86_64 107/165 2024-09-17T03:12:24.380 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-17T03:12:24.474 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-17T03:12:24.501 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-17T03:12:24.534 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-17T03:12:24.859 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-17T03:12:24.924 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : librbd1-2:19.3.0-4989.gce1b342b.el9.x86_64 102/165 2024-09-17T03:12:25.013 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librbd1-2:19.3.0-4989.gce1b342b.el9.x86_64 102/165 2024-09-17T03:12:25.052 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rbd-2:19.3.0-4989.gce1b342b.el9.x86_64 103/165 2024-09-17T03:12:25.094 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-17T03:12:25.601 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-17T03:12:25.633 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librgw2-2:19.3.0-4989.gce1b342b.el9.x86_64 106/165 2024-09-17T03:12:25.705 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librgw2-2:19.3.0-4989.gce1b342b.el9.x86_64 106/165 2024-09-17T03:12:25.727 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rgw-2:19.3.0-4989.gce1b342b.el9.x86_64 107/165 2024-09-17T03:12:26.080 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 108/165 2024-09-17T03:12:26.116 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 108/165 2024-09-17T03:12:26.688 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 108/165 2024-09-17T03:12:26.728 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-base-2:19.3.0-4989.gce1b342b.el9.x86_64 109/165 2024-09-17T03:12:26.823 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-base-2:19.3.0-4989.gce1b342b.el9.x86_64 109/165 2024-09-17T03:12:26.823 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T03:12:26.823 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T03:12:26.823 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:12:26.864 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-selinux-2:19.3.0-4989.gce1b342b.el9.x86_64 110/165 2024-09-17T03:12:28.038 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 108/165 2024-09-17T03:12:28.073 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 108/165 2024-09-17T03:12:28.654 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 108/165 2024-09-17T03:12:28.693 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-base-2:19.3.0-4989.gce1b342b.el9.x86_64 109/165 2024-09-17T03:12:28.776 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-base-2:19.3.0-4989.gce1b342b.el9.x86_64 109/165 2024-09-17T03:12:28.776 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-17T03:12:28.776 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-17T03:12:28.776 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:12:28.811 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-selinux-2:19.3.0-4989.gce1b342b.el9.x86_64 110/165 2024-09-17T03:12:38.584 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4989.gce1b342b.el9.x86_64 110/165 2024-09-17T03:12:38.584 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /sys 2024-09-17T03:12:38.584 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /proc 2024-09-17T03:12:38.584 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /mnt 2024-09-17T03:12:38.584 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /var/tmp 2024-09-17T03:12:38.584 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /home 2024-09-17T03:12:38.584 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /root 2024-09-17T03:12:38.584 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /tmp 2024-09-17T03:12:38.584 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:12:38.767 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-volume-2:19.3.0-4989.gce1b342b.el9.noarch 111/165 2024-09-17T03:12:38.784 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-volume-2:19.3.0-4989.gce1b342b.el9.noarch 111/165 2024-09-17T03:12:38.784 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T03:12:38.784 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T03:12:38.785 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:12:40.370 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-osd-2:19.3.0-4989.gce1b342b.el9.x86_64 112/165 2024-09-17T03:12:40.397 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-osd-2:19.3.0-4989.gce1b342b.el9.x86_64 112/165 2024-09-17T03:12:40.398 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T03:12:40.398 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T03:12:40.398 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T03:12:40.398 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T03:12:40.398 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:12:40.647 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mds-2:19.3.0-4989.gce1b342b.el9.x86_64 113/165 2024-09-17T03:12:40.671 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mds-2:19.3.0-4989.gce1b342b.el9.x86_64 113/165 2024-09-17T03:12:40.671 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T03:12:40.671 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T03:12:40.671 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T03:12:40.671 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T03:12:40.672 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:12:40.756 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4989.gce1b342b.el9.x86_64 110/165 2024-09-17T03:12:40.756 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /sys 2024-09-17T03:12:40.756 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /proc 2024-09-17T03:12:40.756 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /mnt 2024-09-17T03:12:40.756 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /var/tmp 2024-09-17T03:12:40.756 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /home 2024-09-17T03:12:40.756 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /root 2024-09-17T03:12:40.756 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /tmp 2024-09-17T03:12:40.757 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:12:40.945 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-volume-2:19.3.0-4989.gce1b342b.el9.noarch 111/165 2024-09-17T03:12:40.964 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-volume-2:19.3.0-4989.gce1b342b.el9.noarch 111/165 2024-09-17T03:12:40.964 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T03:12:40.964 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-17T03:12:40.964 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:12:41.165 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mon-2:19.3.0-4989.gce1b342b.el9.x86_64 114/165 2024-09-17T03:12:41.189 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mon-2:19.3.0-4989.gce1b342b.el9.x86_64 114/165 2024-09-17T03:12:41.189 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T03:12:41.189 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T03:12:41.189 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T03:12:41.189 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T03:12:41.189 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:12:41.295 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-17T03:12:41.343 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-17T03:12:41.399 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-17T03:12:41.583 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-17T03:12:41.632 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-17T03:12:41.701 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-17T03:12:41.789 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-17T03:12:41.927 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-17T03:12:41.975 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-17T03:12:42.006 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-17T03:12:42.034 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-17T03:12:42.431 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 126/165 2024-09-17T03:12:42.458 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 126/165 2024-09-17T03:12:42.490 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4989.gce1b342b.e 127/165 2024-09-17T03:12:42.556 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4989.gce1b342b. 128/165 2024-09-17T03:12:42.557 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-osd-2:19.3.0-4989.gce1b342b.el9.x86_64 112/165 2024-09-17T03:12:42.581 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4989.gce1b342b.el9.noa 129/165 2024-09-17T03:12:42.583 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-osd-2:19.3.0-4989.gce1b342b.el9.x86_64 112/165 2024-09-17T03:12:42.583 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T03:12:42.583 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-17T03:12:42.584 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T03:12:42.584 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-17T03:12:42.584 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:12:42.815 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mds-2:19.3.0-4989.gce1b342b.el9.x86_64 113/165 2024-09-17T03:12:42.839 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mds-2:19.3.0-4989.gce1b342b.el9.x86_64 113/165 2024-09-17T03:12:42.839 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T03:12:42.839 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-17T03:12:42.839 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T03:12:42.840 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-17T03:12:42.840 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:12:43.317 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mon-2:19.3.0-4989.gce1b342b.el9.x86_64 114/165 2024-09-17T03:12:43.340 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mon-2:19.3.0-4989.gce1b342b.el9.x86_64 114/165 2024-09-17T03:12:43.340 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T03:12:43.340 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-17T03:12:43.340 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T03:12:43.340 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-17T03:12:43.340 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:12:43.446 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-17T03:12:43.504 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-17T03:12:43.568 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-17T03:12:43.751 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-17T03:12:43.809 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-17T03:12:43.894 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-17T03:12:43.991 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-17T03:12:44.140 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-17T03:12:44.184 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-17T03:12:44.216 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-17T03:12:44.243 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-17T03:12:44.674 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 126/165 2024-09-17T03:12:44.700 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 126/165 2024-09-17T03:12:44.732 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4989.gce1b342b.e 127/165 2024-09-17T03:12:44.797 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4989.gce1b342b. 128/165 2024-09-17T03:12:44.823 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4989.gce1b342b.el9.noa 129/165 2024-09-17T03:12:45.737 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4989.gce1b342b.el9.noa 129/165 2024-09-17T03:12:45.787 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4989.gce1b342b.el9.n 130/165 2024-09-17T03:12:46.744 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4989.gce1b342b.el9.n 130/165 2024-09-17T03:12:46.779 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4989.gce1 131/165 2024-09-17T03:12:46.891 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4989.gce1 131/165 2024-09-17T03:12:47.062 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4989.gce1b342b.el 132/165 2024-09-17T03:12:47.090 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-2:19.3.0-4989.gce1b342b.el9.x86_64 133/165 2024-09-17T03:12:47.114 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4989.gce1b342b.el9.x86_64 133/165 2024-09-17T03:12:47.114 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T03:12:47.114 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T03:12:47.114 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T03:12:47.114 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T03:12:47.114 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:12:47.165 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-rook-2:19.3.0-4989.gce1b342b.el9.noarch 134/165 2024-09-17T03:12:47.177 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4989.gce1b342b.el9.noarch 134/165 2024-09-17T03:12:47.275 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-2:19.3.0-4989.gce1b342b.el9.x86_64 135/165 2024-09-17T03:12:47.339 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 136/165 2024-09-17T03:12:48.011 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4989.gce1b342b.el9.noa 129/165 2024-09-17T03:12:48.063 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4989.gce1b342b.el9.n 130/165 2024-09-17T03:12:48.322 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-17T03:12:48.351 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-radosgw-2:19.3.0-4989.gce1b342b.el9.x86_64 138/165 2024-09-17T03:12:48.375 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4989.gce1b342b.el9.x86_64 138/165 2024-09-17T03:12:48.375 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T03:12:48.375 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T03:12:48.375 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T03:12:48.375 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T03:12:48.376 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:12:48.427 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4989.gce1b3 139/165 2024-09-17T03:12:48.450 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4989.gce1b3 139/165 2024-09-17T03:12:48.450 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T03:12:48.450 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T03:12:48.450 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:12:48.502 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 140/165 2024-09-17T03:12:48.526 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 140/165 2024-09-17T03:12:48.526 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T03:12:48.526 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-17T03:12:48.526 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-17T03:12:48.526 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-17T03:12:48.526 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:12:48.853 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 141/165 2024-09-17T03:12:48.877 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 141/165 2024-09-17T03:12:48.877 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T03:12:48.877 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T03:12:48.877 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T03:12:48.877 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T03:12:48.877 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:12:49.025 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4989.gce1b342b.el9.n 130/165 2024-09-17T03:12:49.062 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4989.gce1 131/165 2024-09-17T03:12:49.177 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4989.gce1 131/165 2024-09-17T03:12:49.346 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4989.gce1b342b.el 132/165 2024-09-17T03:12:49.373 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-2:19.3.0-4989.gce1b342b.el9.x86_64 133/165 2024-09-17T03:12:49.397 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4989.gce1b342b.el9.x86_64 133/165 2024-09-17T03:12:49.397 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T03:12:49.397 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-17T03:12:49.397 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T03:12:49.397 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-17T03:12:49.397 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:12:49.448 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-rook-2:19.3.0-4989.gce1b342b.el9.noarch 134/165 2024-09-17T03:12:49.460 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4989.gce1b342b.el9.noarch 134/165 2024-09-17T03:12:49.558 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-2:19.3.0-4989.gce1b342b.el9.x86_64 135/165 2024-09-17T03:12:49.613 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 136/165 2024-09-17T03:12:50.575 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-17T03:12:50.609 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-radosgw-2:19.3.0-4989.gce1b342b.el9.x86_64 138/165 2024-09-17T03:12:50.634 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4989.gce1b342b.el9.x86_64 138/165 2024-09-17T03:12:50.634 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T03:12:50.634 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-17T03:12:50.634 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T03:12:50.634 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-17T03:12:50.634 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:12:50.676 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4989.gce1b3 139/165 2024-09-17T03:12:50.700 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4989.gce1b3 139/165 2024-09-17T03:12:50.700 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T03:12:50.700 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-17T03:12:50.700 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:12:50.744 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cephfs-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 140/165 2024-09-17T03:12:50.767 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 140/165 2024-09-17T03:12:51.011 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T03:12:51.011 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-17T03:12:51.011 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-17T03:12:51.011 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-17T03:12:51.012 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:12:51.086 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 141/165 2024-09-17T03:12:51.110 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 141/165 2024-09-17T03:12:51.110 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-17T03:12:51.110 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-17T03:12:51.110 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T03:12:51.110 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-17T03:12:51.110 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:12:52.875 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-test-2:19.3.0-4989.gce1b342b.el9.x86_64 142/165 2024-09-17T03:12:52.916 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 143/165 2024-09-17T03:12:52.956 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-nbd-2:19.3.0-4989.gce1b342b.el9.x86_64 144/165 2024-09-17T03:12:52.989 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-17T03:12:53.015 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-17T03:12:53.206 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-shell-2:19.3.0-4989.gce1b342b.el9.x86_64 147/165 2024-09-17T03:12:53.254 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-17T03:12:53.284 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs-devel-2:19.3.0-4989.gce1b342b.el9.x86_ 149/165 2024-09-17T03:12:53.314 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-17T03:12:53.352 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-17T03:12:53.953 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-17T03:12:53.992 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-17T03:12:54.012 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-17T03:12:54.845 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephfs-top-2:19.3.0-4989.gce1b342b.el9.noarch 153/165 2024-09-17T03:12:54.930 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-17T03:12:54.981 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-17T03:12:55.061 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-17T03:12:55.098 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-test-2:19.3.0-4989.gce1b342b.el9.x86_64 142/165 2024-09-17T03:12:55.100 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-17T03:12:55.126 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-17T03:12:55.140 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 143/165 2024-09-17T03:12:55.151 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-17T03:12:55.151 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-17T03:12:55.180 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-nbd-2:19.3.0-4989.gce1b342b.el9.x86_64 144/165 2024-09-17T03:12:55.212 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-17T03:12:55.235 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-17T03:12:55.235 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-17T03:12:55.239 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-17T03:12:55.321 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-17T03:12:55.344 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-17T03:12:55.350 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-17T03:12:55.350 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-17T03:12:55.418 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cephfs-shell-2:19.3.0-4989.gce1b342b.el9.x86_64 147/165 2024-09-17T03:12:55.469 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-17T03:12:55.499 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcephfs-devel-2:19.3.0-4989.gce1b342b.el9.x86_ 149/165 2024-09-17T03:12:55.529 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-17T03:12:55.529 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-17T03:12:55.530 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-17T03:12:55.575 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-17T03:12:55.614 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-17T03:12:55.687 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-17T03:12:56.142 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-17T03:12:56.181 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-17T03:12:56.201 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-17T03:12:57.029 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cephfs-top-2:19.3.0-4989.gce1b342b.el9.noarch 153/165 2024-09-17T03:12:57.110 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-17T03:12:57.161 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-17T03:12:57.241 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-17T03:12:57.271 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-17T03:12:57.297 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-17T03:12:57.322 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-17T03:12:57.323 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-17T03:12:57.406 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-17T03:12:57.407 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-17T03:12:57.490 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-17T03:12:57.515 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-17T03:12:57.521 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-17T03:12:57.521 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-17T03:12:57.701 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-17T03:12:57.701 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-17T03:12:57.785 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-17T03:12:57.850 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-17T03:13:01.818 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-17T03:13:01.818 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-2:19.3.0-4989.gce1b342b.el9.x86_64 1/165 2024-09-17T03:13:01.818 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-base-2:19.3.0-4989.gce1b342b.el9.x86_64 2/165 2024-09-17T03:13:01.818 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 3/165 2024-09-17T03:13:01.818 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 4/165 2024-09-17T03:13:01.818 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4989.gce1b3 5/165 2024-09-17T03:13:01.818 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mds-2:19.3.0-4989.gce1b342b.el9.x86_64 6/165 2024-09-17T03:13:01.819 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-2:19.3.0-4989.gce1b342b.el9.x86_64 7/165 2024-09-17T03:13:01.819 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mon-2:19.3.0-4989.gce1b342b.el9.x86_64 8/165 2024-09-17T03:13:01.819 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-osd-2:19.3.0-4989.gce1b342b.el9.x86_64 9/165 2024-09-17T03:13:01.819 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-radosgw-2:19.3.0-4989.gce1b342b.el9.x86_64 10/165 2024-09-17T03:13:01.819 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-selinux-2:19.3.0-4989.gce1b342b.el9.x86_64 11/165 2024-09-17T03:13:01.819 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-test-2:19.3.0-4989.gce1b342b.el9.x86_64 12/165 2024-09-17T03:13:01.819 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 13/165 2024-09-17T03:13:01.819 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-shell-2:19.3.0-4989.gce1b342b.el9.x86_64 14/165 2024-09-17T03:13:01.819 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs-devel-2:19.3.0-4989.gce1b342b.el9.x86_ 15/165 2024-09-17T03:13:01.819 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs2-2:19.3.0-4989.gce1b342b.el9.x86_64 16/165 2024-09-17T03:13:01.819 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephsqlite-2:19.3.0-4989.gce1b342b.el9.x86_64 17/165 2024-09-17T03:13:01.820 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados-devel-2:19.3.0-4989.gce1b342b.el9.x86_6 18/165 2024-09-17T03:13:01.820 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libradosstriper1-2:19.3.0-4989.gce1b342b.el9.x86 19/165 2024-09-17T03:13:01.820 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librgw2-2:19.3.0-4989.gce1b342b.el9.x86_64 20/165 2024-09-17T03:13:01.820 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4989.gce1b342b.el 21/165 2024-09-17T03:13:01.820 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-common-2:19.3.0-4989.gce1b342b.el9. 22/165 2024-09-17T03:13:01.820 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cephfs-2:19.3.0-4989.gce1b342b.el9.x86_6 23/165 2024-09-17T03:13:01.820 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rados-2:19.3.0-4989.gce1b342b.el9.x86_64 24/165 2024-09-17T03:13:01.820 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rbd-2:19.3.0-4989.gce1b342b.el9.x86_64 25/165 2024-09-17T03:13:01.820 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rgw-2:19.3.0-4989.gce1b342b.el9.x86_64 26/165 2024-09-17T03:13:01.820 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 27/165 2024-09-17T03:13:01.820 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 28/165 2024-09-17T03:13:01.820 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-nbd-2:19.3.0-4989.gce1b342b.el9.x86_64 29/165 2024-09-17T03:13:01.821 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4989.gce1b342b. 30/165 2024-09-17T03:13:01.821 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4989.gce1b342b.el9.noa 31/165 2024-09-17T03:13:01.821 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4989.gce1b342b.el9.n 32/165 2024-09-17T03:13:01.821 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4989.gce1 33/165 2024-09-17T03:13:01.821 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4989.gce1b342b.el 34/165 2024-09-17T03:13:01.821 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4989.gce1b342b.el9.noarch 35/165 2024-09-17T03:13:01.821 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4989.gce1b342b.e 36/165 2024-09-17T03:13:01.821 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-volume-2:19.3.0-4989.gce1b342b.el9.noarch 37/165 2024-09-17T03:13:01.821 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 38/165 2024-09-17T03:13:01.821 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephfs-top-2:19.3.0-4989.gce1b342b.el9.noarch 39/165 2024-09-17T03:13:01.821 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-17T03:13:01.821 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-17T03:13:01.821 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-17T03:13:01.822 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-17T03:13:01.822 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-17T03:13:01.822 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-17T03:13:01.822 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-17T03:13:01.822 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-17T03:13:01.822 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-17T03:13:01.822 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-17T03:13:01.822 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-17T03:13:01.822 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-17T03:13:01.823 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-17T03:13:01.823 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-17T03:13:01.823 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-17T03:13:01.823 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-17T03:13:01.823 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-17T03:13:01.823 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-17T03:13:01.823 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-17T03:13:01.823 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-17T03:13:01.823 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-17T03:13:01.823 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-17T03:13:01.823 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-17T03:13:01.823 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-17T03:13:01.824 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-17T03:13:01.824 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-17T03:13:01.824 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-17T03:13:01.824 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-17T03:13:01.824 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-17T03:13:01.824 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-17T03:13:01.824 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-17T03:13:01.824 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-17T03:13:01.824 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-17T03:13:01.824 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-17T03:13:01.824 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-17T03:13:01.824 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-17T03:13:01.824 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-17T03:13:01.825 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-17T03:13:01.825 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-17T03:13:01.825 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-17T03:13:01.825 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-17T03:13:01.825 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-17T03:13:01.825 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-17T03:13:01.825 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-17T03:13:01.825 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-17T03:13:01.825 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-17T03:13:01.825 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-17T03:13:01.825 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-17T03:13:01.825 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-17T03:13:01.825 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-17T03:13:01.826 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-17T03:13:01.826 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-17T03:13:01.826 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-17T03:13:01.826 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-17T03:13:01.826 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-17T03:13:01.826 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-17T03:13:01.826 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-17T03:13:01.826 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-17T03:13:01.826 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-17T03:13:01.826 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-17T03:13:01.826 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-17T03:13:01.826 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-17T03:13:01.826 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-17T03:13:01.827 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-17T03:13:01.827 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-17T03:13:01.827 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-17T03:13:01.827 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-17T03:13:01.827 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-17T03:13:01.827 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-17T03:13:01.827 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-17T03:13:01.827 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-17T03:13:01.827 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-17T03:13:01.827 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-17T03:13:01.827 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-17T03:13:01.827 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-17T03:13:01.827 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-17T03:13:01.828 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-17T03:13:01.828 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-17T03:13:01.828 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-17T03:13:01.828 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-17T03:13:01.828 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-17T03:13:01.828 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-17T03:13:01.828 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-17T03:13:01.828 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-17T03:13:01.828 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-17T03:13:01.828 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-17T03:13:01.828 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-17T03:13:01.828 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-17T03:13:01.828 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-17T03:13:01.828 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-17T03:13:01.829 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-17T03:13:01.829 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-17T03:13:01.829 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-17T03:13:01.829 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-17T03:13:01.829 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-17T03:13:01.829 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-17T03:13:01.829 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-17T03:13:01.829 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-17T03:13:01.829 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-17T03:13:01.829 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-17T03:13:01.829 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-17T03:13:01.829 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-17T03:13:01.830 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-17T03:13:01.830 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-17T03:13:01.830 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-17T03:13:01.830 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-17T03:13:01.830 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-17T03:13:01.830 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-17T03:13:01.830 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-17T03:13:01.830 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-17T03:13:01.830 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:19.3.0-4989.gce1b342b.el9.x86_64 150/165 2024-09-17T03:13:01.830 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-17T03:13:01.830 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:19.3.0-4989.gce1b342b.el9.x86_64 152/165 2024-09-17T03:13:01.831 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-17T03:13:01.831 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-17T03:13:01.831 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-17T03:13:01.831 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-17T03:13:01.831 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-17T03:13:01.831 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-17T03:13:01.831 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-17T03:13:01.831 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-17T03:13:01.831 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-17T03:13:01.831 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-17T03:13:01.831 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-17T03:13:01.831 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-17T03:13:04.226 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-17T03:13:04.227 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-2:19.3.0-4989.gce1b342b.el9.x86_64 1/165 2024-09-17T03:13:04.227 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-base-2:19.3.0-4989.gce1b342b.el9.x86_64 2/165 2024-09-17T03:13:04.227 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 3/165 2024-09-17T03:13:04.227 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 4/165 2024-09-17T03:13:04.227 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4989.gce1b3 5/165 2024-09-17T03:13:04.227 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mds-2:19.3.0-4989.gce1b342b.el9.x86_64 6/165 2024-09-17T03:13:04.227 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-2:19.3.0-4989.gce1b342b.el9.x86_64 7/165 2024-09-17T03:13:04.227 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mon-2:19.3.0-4989.gce1b342b.el9.x86_64 8/165 2024-09-17T03:13:04.227 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-osd-2:19.3.0-4989.gce1b342b.el9.x86_64 9/165 2024-09-17T03:13:04.227 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-radosgw-2:19.3.0-4989.gce1b342b.el9.x86_64 10/165 2024-09-17T03:13:04.227 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-selinux-2:19.3.0-4989.gce1b342b.el9.x86_64 11/165 2024-09-17T03:13:04.228 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-test-2:19.3.0-4989.gce1b342b.el9.x86_64 12/165 2024-09-17T03:13:04.228 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cephfs-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 13/165 2024-09-17T03:13:04.228 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cephfs-shell-2:19.3.0-4989.gce1b342b.el9.x86_64 14/165 2024-09-17T03:13:04.228 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcephfs-devel-2:19.3.0-4989.gce1b342b.el9.x86_ 15/165 2024-09-17T03:13:04.228 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcephfs2-2:19.3.0-4989.gce1b342b.el9.x86_64 16/165 2024-09-17T03:13:04.228 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcephsqlite-2:19.3.0-4989.gce1b342b.el9.x86_64 17/165 2024-09-17T03:13:04.228 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados-devel-2:19.3.0-4989.gce1b342b.el9.x86_6 18/165 2024-09-17T03:13:04.228 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libradosstriper1-2:19.3.0-4989.gce1b342b.el9.x86 19/165 2024-09-17T03:13:04.228 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librgw2-2:19.3.0-4989.gce1b342b.el9.x86_64 20/165 2024-09-17T03:13:04.228 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4989.gce1b342b.el 21/165 2024-09-17T03:13:04.228 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-ceph-common-2:19.3.0-4989.gce1b342b.el9. 22/165 2024-09-17T03:13:04.228 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cephfs-2:19.3.0-4989.gce1b342b.el9.x86_6 23/165 2024-09-17T03:13:04.228 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rados-2:19.3.0-4989.gce1b342b.el9.x86_64 24/165 2024-09-17T03:13:04.229 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rbd-2:19.3.0-4989.gce1b342b.el9.x86_64 25/165 2024-09-17T03:13:04.229 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rgw-2:19.3.0-4989.gce1b342b.el9.x86_64 26/165 2024-09-17T03:13:04.229 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 27/165 2024-09-17T03:13:04.241 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 28/165 2024-09-17T03:13:04.241 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-nbd-2:19.3.0-4989.gce1b342b.el9.x86_64 29/165 2024-09-17T03:13:04.241 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4989.gce1b342b. 30/165 2024-09-17T03:13:04.241 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4989.gce1b342b.el9.noa 31/165 2024-09-17T03:13:04.241 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4989.gce1b342b.el9.n 32/165 2024-09-17T03:13:04.241 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4989.gce1 33/165 2024-09-17T03:13:04.241 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4989.gce1b342b.el 34/165 2024-09-17T03:13:04.241 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4989.gce1b342b.el9.noarch 35/165 2024-09-17T03:13:04.241 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4989.gce1b342b.e 36/165 2024-09-17T03:13:04.241 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-volume-2:19.3.0-4989.gce1b342b.el9.noarch 37/165 2024-09-17T03:13:04.242 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 38/165 2024-09-17T03:13:04.242 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cephfs-top-2:19.3.0-4989.gce1b342b.el9.noarch 39/165 2024-09-17T03:13:04.242 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-17T03:13:04.242 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-17T03:13:04.242 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-17T03:13:04.242 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-17T03:13:04.242 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-17T03:13:04.242 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-17T03:13:04.242 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-17T03:13:04.242 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-17T03:13:04.242 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-17T03:13:04.243 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-17T03:13:04.243 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-17T03:13:04.243 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-17T03:13:04.243 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-17T03:13:04.243 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-17T03:13:04.243 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-17T03:13:04.243 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-17T03:13:04.243 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-17T03:13:04.243 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-17T03:13:04.243 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-17T03:13:04.243 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-17T03:13:04.243 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-17T03:13:04.243 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-17T03:13:04.243 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-17T03:13:04.244 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-17T03:13:04.244 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-17T03:13:04.244 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-17T03:13:04.244 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-17T03:13:04.244 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-17T03:13:04.244 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-17T03:13:04.244 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-17T03:13:04.244 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-17T03:13:04.244 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-17T03:13:04.244 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-17T03:13:04.244 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-17T03:13:04.245 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-17T03:13:04.245 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-17T03:13:04.245 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-17T03:13:04.245 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-17T03:13:04.245 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-17T03:13:04.245 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-17T03:13:04.245 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-17T03:13:04.245 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-17T03:13:04.245 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-17T03:13:04.245 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-17T03:13:04.245 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-17T03:13:04.245 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-17T03:13:04.246 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-17T03:13:04.246 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-17T03:13:04.246 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-17T03:13:04.246 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-17T03:13:04.246 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-17T03:13:04.246 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-17T03:13:04.246 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-17T03:13:04.246 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-17T03:13:04.246 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-17T03:13:04.246 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-17T03:13:04.246 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-17T03:13:04.246 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-17T03:13:04.246 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-17T03:13:04.247 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-17T03:13:04.247 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-17T03:13:04.247 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-17T03:13:04.247 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-17T03:13:04.247 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-17T03:13:04.247 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-17T03:13:04.248 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-17T03:13:04.248 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-17T03:13:04.248 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-17T03:13:04.248 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-17T03:13:04.248 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-17T03:13:04.248 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-17T03:13:04.248 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-17T03:13:04.248 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-17T03:13:04.248 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-17T03:13:04.248 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-17T03:13:04.248 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-17T03:13:04.248 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-17T03:13:04.249 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-17T03:13:04.249 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-17T03:13:04.249 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-17T03:13:04.249 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-17T03:13:04.249 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-17T03:13:04.249 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-17T03:13:04.249 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-17T03:13:04.249 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-17T03:13:04.249 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-17T03:13:04.249 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-17T03:13:04.249 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-17T03:13:04.249 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-17T03:13:04.249 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-17T03:13:04.250 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-17T03:13:04.250 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-17T03:13:04.250 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-17T03:13:04.250 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-17T03:13:04.250 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-17T03:13:04.250 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-17T03:13:04.250 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-17T03:13:04.250 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-17T03:13:04.250 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-17T03:13:04.250 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-17T03:13:04.250 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-17T03:13:04.250 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-17T03:13:04.250 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-17T03:13:04.251 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-17T03:13:04.251 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-17T03:13:04.251 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-17T03:13:04.251 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-17T03:13:04.251 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-17T03:13:04.251 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-17T03:13:04.251 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-17T03:13:04.251 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados2-2:19.3.0-4989.gce1b342b.el9.x86_64 150/165 2024-09-17T03:13:04.251 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-17T03:13:04.251 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librbd1-2:19.3.0-4989.gce1b342b.el9.x86_64 152/165 2024-09-17T03:13:04.251 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-17T03:13:04.251 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-17T03:13:04.252 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-17T03:13:04.252 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-17T03:13:04.252 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-17T03:13:04.252 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-17T03:13:04.252 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-17T03:13:04.252 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-17T03:13:04.252 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-17T03:13:04.252 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-17T03:13:04.252 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-17T03:13:04.252 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-17T03:13:04.361 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-17T03:13:04.361 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:04.361 INFO:teuthology.orchestra.run.smithi149.stdout:Upgraded: 2024-09-17T03:13:04.361 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-17T03:13:04.361 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-17T03:13:04.361 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-17T03:13:04.361 INFO:teuthology.orchestra.run.smithi149.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-17T03:13:04.361 INFO:teuthology.orchestra.run.smithi149.stdout: librados2-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.361 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.361 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-17T03:13:04.362 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-17T03:13:04.362 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-09-17T03:13:04.362 INFO:teuthology.orchestra.run.smithi149.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T03:13:04.362 INFO:teuthology.orchestra.run.smithi149.stdout: automake-1.16.2-8.el9.noarch 2024-09-17T03:13:04.362 INFO:teuthology.orchestra.run.smithi149.stdout: bison-3.7.4-5.el9.x86_64 2024-09-17T03:13:04.362 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T03:13:04.362 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.362 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.362 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.362 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.362 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T03:13:04.362 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.362 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.363 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.363 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T03:13:04.363 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T03:13:04.363 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T03:13:04.363 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T03:13:04.363 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T03:13:04.363 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.363 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.363 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T03:13:04.363 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.363 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.363 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.364 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T03:13:04.364 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T03:13:04.364 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.364 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-shell-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.364 INFO:teuthology.orchestra.run.smithi149.stdout: cephfs-top-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T03:13:04.364 INFO:teuthology.orchestra.run.smithi149.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-17T03:13:04.364 INFO:teuthology.orchestra.run.smithi149.stdout: flex-2.6.4-9.el9.x86_64 2024-09-17T03:13:04.364 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T03:13:04.364 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T03:13:04.364 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T03:13:04.364 INFO:teuthology.orchestra.run.smithi149.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-17T03:13:04.364 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T03:13:04.364 INFO:teuthology.orchestra.run.smithi149.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T03:13:04.365 INFO:teuthology.orchestra.run.smithi149.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-17T03:13:04.365 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T03:13:04.365 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T03:13:04.365 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.365 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.365 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.365 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T03:13:04.365 INFO:teuthology.orchestra.run.smithi149.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-17T03:13:04.365 INFO:teuthology.orchestra.run.smithi149.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T03:13:04.365 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T03:13:04.365 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T03:13:04.365 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T03:13:04.365 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.366 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.366 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T03:13:04.366 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.366 INFO:teuthology.orchestra.run.smithi149.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-17T03:13:04.366 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T03:13:04.366 INFO:teuthology.orchestra.run.smithi149.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-17T03:13:04.366 INFO:teuthology.orchestra.run.smithi149.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-17T03:13:04.366 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T03:13:04.366 INFO:teuthology.orchestra.run.smithi149.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T03:13:04.366 INFO:teuthology.orchestra.run.smithi149.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T03:13:04.366 INFO:teuthology.orchestra.run.smithi149.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T03:13:04.366 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T03:13:04.366 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T03:13:04.367 INFO:teuthology.orchestra.run.smithi149.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-17T03:13:04.367 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T03:13:04.367 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-13.16-1.el9.x86_64 2024-09-17T03:13:04.367 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-17T03:13:04.367 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-17T03:13:04.367 INFO:teuthology.orchestra.run.smithi149.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-17T03:13:04.367 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T03:13:04.367 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T03:13:04.367 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T03:13:04.367 INFO:teuthology.orchestra.run.smithi149.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-17T03:13:04.367 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T03:13:04.367 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T03:13:04.367 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T03:13:04.368 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.368 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.368 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.368 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T03:13:04.368 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T03:13:04.368 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T03:13:04.368 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T03:13:04.368 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-17T03:13:04.368 INFO:teuthology.orchestra.run.smithi149.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-17T03:13:04.368 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T03:13:04.368 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T03:13:04.368 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T03:13:04.368 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T03:13:04.368 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T03:13:04.369 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T03:13:04.369 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T03:13:04.369 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T03:13:04.369 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T03:13:04.369 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T03:13:04.369 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T03:13:04.369 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T03:13:04.369 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T03:13:04.369 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T03:13:04.369 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T03:13:04.369 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T03:13:04.369 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T03:13:04.370 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T03:13:04.370 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T03:13:04.370 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T03:13:04.370 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T03:13:04.370 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T03:13:04.370 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T03:13:04.370 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T03:13:04.370 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T03:13:04.370 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T03:13:04.370 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T03:13:04.370 INFO:teuthology.orchestra.run.smithi149.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T03:13:04.370 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T03:13:04.371 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T03:13:04.371 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T03:13:04.371 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T03:13:04.371 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-17T03:13:04.371 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T03:13:04.371 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.371 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.371 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T03:13:04.371 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T03:13:04.371 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T03:13:04.371 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.371 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T03:13:04.371 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T03:13:04.372 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T03:13:04.372 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T03:13:04.372 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T03:13:04.372 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T03:13:04.372 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T03:13:04.372 INFO:teuthology.orchestra.run.smithi149.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-17T03:13:04.372 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T03:13:04.372 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T03:13:04.372 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T03:13:04.372 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T03:13:04.372 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T03:13:04.372 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T03:13:04.372 INFO:teuthology.orchestra.run.smithi149.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T03:13:04.373 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.373 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.373 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:04.373 INFO:teuthology.orchestra.run.smithi149.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T03:13:04.373 INFO:teuthology.orchestra.run.smithi149.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T03:13:04.373 INFO:teuthology.orchestra.run.smithi149.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T03:13:04.373 INFO:teuthology.orchestra.run.smithi149.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-17T03:13:04.373 INFO:teuthology.orchestra.run.smithi149.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-17T03:13:04.373 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T03:13:04.374 INFO:teuthology.orchestra.run.smithi149.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-17T03:13:04.374 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:04.374 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-09-17T03:13:04.808 DEBUG:teuthology.parallel:result is None 2024-09-17T03:13:06.779 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-17T03:13:06.779 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:06.779 INFO:teuthology.orchestra.run.smithi087.stdout:Upgraded: 2024-09-17T03:13:06.779 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-17T03:13:06.780 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-17T03:13:06.780 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-17T03:13:06.780 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-17T03:13:06.780 INFO:teuthology.orchestra.run.smithi087.stdout: librados2-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.780 INFO:teuthology.orchestra.run.smithi087.stdout: librbd1-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.780 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-17T03:13:06.780 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-17T03:13:06.780 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2024-09-17T03:13:06.780 INFO:teuthology.orchestra.run.smithi087.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-17T03:13:06.780 INFO:teuthology.orchestra.run.smithi087.stdout: automake-1.16.2-8.el9.noarch 2024-09-17T03:13:06.780 INFO:teuthology.orchestra.run.smithi087.stdout: bison-3.7.4-5.el9.x86_64 2024-09-17T03:13:06.780 INFO:teuthology.orchestra.run.smithi087.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-17T03:13:06.781 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.781 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-base-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.781 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.781 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.781 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-grafana-dashboards-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T03:13:06.781 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-immutable-object-cache-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.781 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mds-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.781 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.781 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T03:13:06.781 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-dashboard-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T03:13:06.781 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T03:13:06.781 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-modules-core-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T03:13:06.782 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-rook-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T03:13:06.782 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mon-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.782 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-osd-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.782 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-prometheus-alerts-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T03:13:06.782 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-radosgw-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.782 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-selinux-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.782 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-test-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.782 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-volume-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T03:13:06.782 INFO:teuthology.orchestra.run.smithi087.stdout: cephadm-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T03:13:06.782 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.783 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-shell-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.783 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-top-2:19.3.0-4989.gce1b342b.el9.noarch 2024-09-17T03:13:06.783 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-17T03:13:06.783 INFO:teuthology.orchestra.run.smithi087.stdout: flex-2.6.4-9.el9.x86_64 2024-09-17T03:13:06.783 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-17T03:13:06.783 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-17T03:13:06.783 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-17T03:13:06.783 INFO:teuthology.orchestra.run.smithi087.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-17T03:13:06.783 INFO:teuthology.orchestra.run.smithi087.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-17T03:13:06.783 INFO:teuthology.orchestra.run.smithi087.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-17T03:13:06.784 INFO:teuthology.orchestra.run.smithi087.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-17T03:13:06.784 INFO:teuthology.orchestra.run.smithi087.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-17T03:13:06.784 INFO:teuthology.orchestra.run.smithi087.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-17T03:13:06.784 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs-devel-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.784 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs2-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.784 INFO:teuthology.orchestra.run.smithi087.stdout: libcephsqlite-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.784 INFO:teuthology.orchestra.run.smithi087.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-17T03:13:06.784 INFO:teuthology.orchestra.run.smithi087.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-17T03:13:06.784 INFO:teuthology.orchestra.run.smithi087.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-17T03:13:06.784 INFO:teuthology.orchestra.run.smithi087.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-17T03:13:06.785 INFO:teuthology.orchestra.run.smithi087.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-17T03:13:06.785 INFO:teuthology.orchestra.run.smithi087.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-17T03:13:06.785 INFO:teuthology.orchestra.run.smithi087.stdout: librados-devel-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.785 INFO:teuthology.orchestra.run.smithi087.stdout: libradosstriper1-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.785 INFO:teuthology.orchestra.run.smithi087.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-17T03:13:06.785 INFO:teuthology.orchestra.run.smithi087.stdout: librgw2-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.785 INFO:teuthology.orchestra.run.smithi087.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-17T03:13:06.785 INFO:teuthology.orchestra.run.smithi087.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-17T03:13:06.785 INFO:teuthology.orchestra.run.smithi087.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-17T03:13:06.785 INFO:teuthology.orchestra.run.smithi087.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-17T03:13:06.785 INFO:teuthology.orchestra.run.smithi087.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-17T03:13:06.786 INFO:teuthology.orchestra.run.smithi087.stdout: lua-5.4.4-4.el9.x86_64 2024-09-17T03:13:06.786 INFO:teuthology.orchestra.run.smithi087.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-17T03:13:06.786 INFO:teuthology.orchestra.run.smithi087.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-17T03:13:06.786 INFO:teuthology.orchestra.run.smithi087.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-17T03:13:06.786 INFO:teuthology.orchestra.run.smithi087.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-17T03:13:06.786 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-17T03:13:06.786 INFO:teuthology.orchestra.run.smithi087.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-17T03:13:06.786 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-13.16-1.el9.x86_64 2024-09-17T03:13:06.786 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-17T03:13:06.786 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-17T03:13:06.786 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-17T03:13:06.786 INFO:teuthology.orchestra.run.smithi087.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-17T03:13:06.786 INFO:teuthology.orchestra.run.smithi087.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-17T03:13:06.787 INFO:teuthology.orchestra.run.smithi087.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-17T03:13:06.787 INFO:teuthology.orchestra.run.smithi087.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-17T03:13:06.787 INFO:teuthology.orchestra.run.smithi087.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-17T03:13:06.787 INFO:teuthology.orchestra.run.smithi087.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-17T03:13:06.787 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-17T03:13:06.787 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-argparse-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.787 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-common-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.787 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cephfs-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.787 INFO:teuthology.orchestra.run.smithi087.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-17T03:13:06.787 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-17T03:13:06.787 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-17T03:13:06.787 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-17T03:13:06.788 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-17T03:13:06.788 INFO:teuthology.orchestra.run.smithi087.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-17T03:13:06.788 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-17T03:13:06.788 INFO:teuthology.orchestra.run.smithi087.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-17T03:13:06.788 INFO:teuthology.orchestra.run.smithi087.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-17T03:13:06.788 INFO:teuthology.orchestra.run.smithi087.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-17T03:13:06.788 INFO:teuthology.orchestra.run.smithi087.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-17T03:13:06.788 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-17T03:13:06.788 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-17T03:13:06.788 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-17T03:13:06.788 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-17T03:13:06.788 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-17T03:13:06.788 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-17T03:13:06.789 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-17T03:13:06.789 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-17T03:13:06.789 INFO:teuthology.orchestra.run.smithi087.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-17T03:13:06.789 INFO:teuthology.orchestra.run.smithi087.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-17T03:13:06.789 INFO:teuthology.orchestra.run.smithi087.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-17T03:13:06.789 INFO:teuthology.orchestra.run.smithi087.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-17T03:13:06.789 INFO:teuthology.orchestra.run.smithi087.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-17T03:13:06.789 INFO:teuthology.orchestra.run.smithi087.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-17T03:13:06.789 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-17T03:13:06.789 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-17T03:13:06.789 INFO:teuthology.orchestra.run.smithi087.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-17T03:13:06.789 INFO:teuthology.orchestra.run.smithi087.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-17T03:13:06.789 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-17T03:13:06.790 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ply-3.11-14.el9.noarch 2024-09-17T03:13:06.790 INFO:teuthology.orchestra.run.smithi087.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-17T03:13:06.790 INFO:teuthology.orchestra.run.smithi087.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-17T03:13:06.790 INFO:teuthology.orchestra.run.smithi087.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-17T03:13:06.790 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-17T03:13:06.790 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-17T03:13:06.790 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-17T03:13:06.790 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-17T03:13:06.790 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-17T03:13:06.790 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-17T03:13:06.790 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rados-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.790 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rbd-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.791 INFO:teuthology.orchestra.run.smithi087.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-17T03:13:06.791 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-17T03:13:06.791 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-17T03:13:06.791 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rgw-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.791 INFO:teuthology.orchestra.run.smithi087.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-17T03:13:06.791 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-17T03:13:06.791 INFO:teuthology.orchestra.run.smithi087.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-17T03:13:06.791 INFO:teuthology.orchestra.run.smithi087.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-17T03:13:06.791 INFO:teuthology.orchestra.run.smithi087.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-17T03:13:06.791 INFO:teuthology.orchestra.run.smithi087.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-17T03:13:06.791 INFO:teuthology.orchestra.run.smithi087.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-17T03:13:06.791 INFO:teuthology.orchestra.run.smithi087.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-17T03:13:06.791 INFO:teuthology.orchestra.run.smithi087.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-17T03:13:06.792 INFO:teuthology.orchestra.run.smithi087.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-17T03:13:06.792 INFO:teuthology.orchestra.run.smithi087.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-17T03:13:06.792 INFO:teuthology.orchestra.run.smithi087.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-17T03:13:06.792 INFO:teuthology.orchestra.run.smithi087.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-17T03:13:06.792 INFO:teuthology.orchestra.run.smithi087.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-17T03:13:06.792 INFO:teuthology.orchestra.run.smithi087.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-17T03:13:06.792 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-fuse-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.792 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-mirror-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.792 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-nbd-2:19.3.0-4989.gce1b342b.el9.x86_64 2024-09-17T03:13:06.792 INFO:teuthology.orchestra.run.smithi087.stdout: re2-1:20211101-20.el9.x86_64 2024-09-17T03:13:06.792 INFO:teuthology.orchestra.run.smithi087.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-17T03:13:06.792 INFO:teuthology.orchestra.run.smithi087.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-17T03:13:06.792 INFO:teuthology.orchestra.run.smithi087.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-17T03:13:06.793 INFO:teuthology.orchestra.run.smithi087.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-17T03:13:06.793 INFO:teuthology.orchestra.run.smithi087.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-17T03:13:06.793 INFO:teuthology.orchestra.run.smithi087.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-17T03:13:06.793 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:06.793 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2024-09-17T03:13:07.195 DEBUG:teuthology.parallel:result is None 2024-09-17T03:13:07.196 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T03:13:07.347 DEBUG:teuthology.orchestra.run.smithi087:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T03:13:07.369 INFO:teuthology.orchestra.run.smithi087.stdout:19.3.0-4989.gce1b342b.el9 2024-09-17T03:13:07.370 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4989.gce1b342b.el9 2024-09-17T03:13:07.370 INFO:teuthology.task.install:The correct ceph version 19.3.0-4989.gce1b342b is installed. 2024-09-17T03:13:07.372 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=ce1b342bf8c6349a8c90cd607f83bff2443f21cf 2024-09-17T03:13:07.522 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-17T03:13:07.548 INFO:teuthology.orchestra.run.smithi149.stdout:19.3.0-4989.gce1b342b.el9 2024-09-17T03:13:07.549 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4989.gce1b342b.el9 2024-09-17T03:13:07.549 INFO:teuthology.task.install:The correct ceph version 19.3.0-4989.gce1b342b is installed. 2024-09-17T03:13:07.551 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-17T03:13:07.551 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:13:07.551 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T03:13:07.578 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:13:07.578 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-17T03:13:07.619 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-17T03:13:07.619 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:13:07.619 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T03:13:07.645 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T03:13:07.710 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:13:07.710 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/daemon-helper 2024-09-17T03:13:07.739 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-17T03:13:07.806 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-17T03:13:07.806 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:13:07.806 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T03:13:07.831 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T03:13:07.895 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:13:07.895 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-17T03:13:07.922 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-17T03:13:07.988 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-17T03:13:07.989 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:13:07.989 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T03:13:08.013 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T03:13:08.078 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:13:08.078 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/stdin-killer 2024-09-17T03:13:08.103 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-17T03:13:08.169 INFO:teuthology.run_tasks:Running task ceph... 2024-09-17T03:13:08.257 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-17T03:13:08.257 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 777 /var/log/ceph 2024-09-17T03:13:08.259 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 777 /var/log/ceph 2024-09-17T03:13:08.284 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-17T03:13:08.284 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T03:13:08.325 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-17T03:13:08.349 INFO:tasks.ceph:Creating extra log directories... 2024-09-17T03:13:08.350 DEBUG:teuthology.orchestra.run.smithi087:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T03:13:08.390 DEBUG:teuthology.orchestra.run.smithi149:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-17T03:13:08.420 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-17T03:13:08.420 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-17T03:13:08.421 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-17T03:13:08.421 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:13:08.421 DEBUG:teuthology.orchestra.run.smithi087:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-17T03:13:08.468 DEBUG:teuthology.orchestra.run.smithi087:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-17T03:13:08.577 DEBUG:teuthology.orchestra.run.smithi087:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-17T03:13:08.601 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:13:08.601 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-17T03:13:08.616 DEBUG:teuthology.orchestra.run.smithi149:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-17T03:13:08.727 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-17T03:13:08.757 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-17T03:13:08.757 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-17T03:13:08.757 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908414', 'branch': 'wip-pdonnell-testing-20240916.200549-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/pool-perm}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7908414', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240916.200549-debug'}, 'ceph': {'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240916.200549-debug', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 90, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 476, 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'sleep_before_teardown': 0, 'subset': '320/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240916.200549-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi087.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDEe/9gXlj72LBxMUaRxcRamI5bYmWqTB17Qrk/m225fz+JvPGAnYsBZVhIZGv60j2Tfdf6a9+bgBfWmfYx/ghc=', 'smithi149.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAzVyyv67NP4OhKKq5Ej7UvLDbw+/NEWEzT8Q53yy9Z1BpdB5yWNM2LI1nik3MDr04az/b+66ERM6ibsfxvEFO0='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf'}}, {'ceph': {'create_rbd_pool': False, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': 'ce1b342bf8c6349a8c90cd607f83bff2443f21cf', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_pool_perm']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_02:09:51', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-17T03:13:08.758 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T03:13:08.760 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-17T03:13:08.808 DEBUG:teuthology.orchestra.run.smithi087:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T03:13:08.814 DEBUG:teuthology.orchestra.run.smithi149:> sudo install -d -m0777 -- /var/run/ceph 2024-09-17T03:13:08.872 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:13:08.872 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T03:13:08.892 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T03:13:08.892 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_1 2024-09-17T03:13:08.947 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T03:13:08.947 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T03:13:08.947 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-17T03:13:08.947 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T03:13:08.947 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T03:13:08.947 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-09-17 03:12:58.229484430 +0000 2024-09-17T03:13:08.947 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-09-17 03:10:57.832625864 +0000 2024-09-17T03:13:08.948 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-09-17 03:10:57.832625864 +0000 2024-09-17T03:13:08.948 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-09-17 03:10:57.832625864 +0000 2024-09-17T03:13:08.948 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T03:13:09.011 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-09-17T03:13:09.011 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-09-17T03:13:09.011 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000204164 s, 2.5 MB/s 2024-09-17T03:13:09.012 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T03:13:09.069 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_2 2024-09-17T03:13:09.124 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T03:13:09.124 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T03:13:09.124 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-17T03:13:09.124 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T03:13:09.124 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T03:13:09.125 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-09-17 03:12:58.229484430 +0000 2024-09-17T03:13:09.125 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-09-17 03:10:58.163617886 +0000 2024-09-17T03:13:09.125 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-09-17 03:10:58.163617886 +0000 2024-09-17T03:13:09.125 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-09-17 03:10:58.163617886 +0000 2024-09-17T03:13:09.125 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T03:13:09.188 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-09-17T03:13:09.188 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-09-17T03:13:09.188 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000224671 s, 2.3 MB/s 2024-09-17T03:13:09.190 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T03:13:09.245 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_3 2024-09-17T03:13:09.300 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T03:13:09.301 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T03:13:09.301 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-17T03:13:09.301 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T03:13:09.301 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T03:13:09.301 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-09-17 03:12:58.230484406 +0000 2024-09-17T03:13:09.301 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-09-17 03:10:58.488610053 +0000 2024-09-17T03:13:09.301 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-09-17 03:10:58.488610053 +0000 2024-09-17T03:13:09.301 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-09-17 03:10:58.488610053 +0000 2024-09-17T03:13:09.301 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T03:13:09.363 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-09-17T03:13:09.363 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-09-17T03:13:09.364 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000174067 s, 2.9 MB/s 2024-09-17T03:13:09.365 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T03:13:09.420 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_4 2024-09-17T03:13:09.475 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T03:13:09.476 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T03:13:09.476 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-17T03:13:09.476 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T03:13:09.476 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T03:13:09.476 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-09-17 03:12:58.230484406 +0000 2024-09-17T03:13:09.476 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-09-17 03:10:58.816602148 +0000 2024-09-17T03:13:09.476 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-09-17 03:10:58.816602148 +0000 2024-09-17T03:13:09.476 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-09-17 03:10:58.816602148 +0000 2024-09-17T03:13:09.476 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T03:13:09.538 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-09-17T03:13:09.538 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-09-17T03:13:09.539 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.00018042 s, 2.8 MB/s 2024-09-17T03:13:09.540 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T03:13:09.595 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T03:13:09.595 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:13:09.596 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/scratch_devs of=/dev/stdout 2024-09-17T03:13:09.613 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-17T03:13:09.613 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_1 2024-09-17T03:13:09.670 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-17T03:13:09.670 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T03:13:09.670 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 798 Links: 1 2024-09-17T03:13:09.670 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T03:13:09.670 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T03:13:09.670 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-09-17 03:12:56.050735070 +0000 2024-09-17T03:13:09.670 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-09-17 03:11:05.319249723 +0000 2024-09-17T03:13:09.670 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-09-17 03:11:05.319249723 +0000 2024-09-17T03:13:09.670 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-09-17 03:11:05.319249723 +0000 2024-09-17T03:13:09.671 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-17T03:13:09.735 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-09-17T03:13:09.735 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-09-17T03:13:09.735 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000188484 s, 2.7 MB/s 2024-09-17T03:13:09.737 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-17T03:13:09.794 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_2 2024-09-17T03:13:09.852 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-17T03:13:09.852 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T03:13:09.852 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 816 Links: 1 2024-09-17T03:13:09.852 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T03:13:09.852 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T03:13:09.852 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-09-17 03:12:56.050735070 +0000 2024-09-17T03:13:09.852 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-09-17 03:11:05.646243385 +0000 2024-09-17T03:13:09.853 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-09-17 03:11:05.646243385 +0000 2024-09-17T03:13:09.853 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-09-17 03:11:05.646243385 +0000 2024-09-17T03:13:09.853 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-17T03:13:09.916 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-09-17T03:13:09.916 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-09-17T03:13:09.916 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000210623 s, 2.4 MB/s 2024-09-17T03:13:09.917 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-17T03:13:09.973 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_3 2024-09-17T03:13:10.028 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-17T03:13:10.029 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T03:13:10.029 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 835 Links: 1 2024-09-17T03:13:10.029 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T03:13:10.029 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T03:13:10.029 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-09-17 03:12:56.051735051 +0000 2024-09-17T03:13:10.029 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-09-17 03:11:05.968237144 +0000 2024-09-17T03:13:10.029 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-09-17 03:11:05.968237144 +0000 2024-09-17T03:13:10.029 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-09-17 03:11:05.968237144 +0000 2024-09-17T03:13:10.029 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-17T03:13:10.092 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-09-17T03:13:10.092 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-09-17T03:13:10.092 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000200254 s, 2.6 MB/s 2024-09-17T03:13:10.093 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-17T03:13:10.149 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_4 2024-09-17T03:13:10.204 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-17T03:13:10.205 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-17T03:13:10.205 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 856 Links: 1 2024-09-17T03:13:10.205 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-17T03:13:10.205 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-09-17T03:13:10.205 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-09-17 03:12:56.051735051 +0000 2024-09-17T03:13:10.205 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-09-17 03:11:06.287230961 +0000 2024-09-17T03:13:10.205 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-09-17 03:11:06.287230961 +0000 2024-09-17T03:13:10.205 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-09-17 03:11:06.287230961 +0000 2024-09-17T03:13:10.205 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-17T03:13:10.268 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-09-17T03:13:10.268 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-09-17T03:13:10.268 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000230415 s, 2.2 MB/s 2024-09-17T03:13:10.269 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-17T03:13:10.325 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T03:13:10.326 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi087.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi149.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-17T03:13:10.326 INFO:tasks.ceph:Generating config... 2024-09-17T03:13:10.327 INFO:tasks.ceph:[client] pid file = 2024-09-17T03:13:10.327 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-17T03:13:10.327 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-17T03:13:10.327 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-17T03:13:10.327 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-17T03:13:10.327 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-17T03:13:10.328 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-17T03:13:10.328 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-17T03:13:10.328 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-17T03:13:10.328 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-17T03:13:10.328 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-17T03:13:10.328 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-17T03:13:10.328 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-17T03:13:10.328 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-17T03:13:10.328 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-17T03:13:10.328 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-17T03:13:10.328 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-17T03:13:10.328 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-17T03:13:10.328 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-17T03:13:10.328 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-17T03:13:10.329 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-17T03:13:10.329 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-17T03:13:10.329 INFO:tasks.ceph:Setting up mon.a... 2024-09-17T03:13:10.329 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-17T03:13:10.387 INFO:teuthology.orchestra.run.smithi087.stdout:creating /etc/ceph/ceph.keyring 2024-09-17T03:13:10.390 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-17T03:13:10.443 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T03:13:10.467 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.87'), ('mon.b', '172.21.15.149'), ('mon.c', '[v2:172.21.15.149:3301,v1:172.21.15.149:6790]')] 2024-09-17T03:13:10.468 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.87,172.21.15.149,[v2:172.21.15.149:3301,v1:172.21.15.149:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-17T03:13:10.469 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:13:10.469 DEBUG:teuthology.orchestra.run.smithi087:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T03:13:10.523 DEBUG:teuthology.orchestra.run.smithi087:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.87 --add b 172.21.15.149 --addv c '[v2:172.21.15.149:3301,v1:172.21.15.149:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:13:10.620 INFO:teuthology.orchestra.run.smithi087.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:13:10.620 INFO:teuthology.orchestra.run.smithi087.stdout:monmaptool: generated fsid 7b2553aa-e562-4da8-b175-8e91f460da55 2024-09-17T03:13:10.620 INFO:teuthology.orchestra.run.smithi087.stdout:setting min_mon_release = quincy 2024-09-17T03:13:10.620 INFO:teuthology.orchestra.run.smithi087.stdout:epoch 0 2024-09-17T03:13:10.620 INFO:teuthology.orchestra.run.smithi087.stdout:fsid 7b2553aa-e562-4da8-b175-8e91f460da55 2024-09-17T03:13:10.620 INFO:teuthology.orchestra.run.smithi087.stdout:last_changed 2024-09-17T03:13:10.622351+0000 2024-09-17T03:13:10.620 INFO:teuthology.orchestra.run.smithi087.stdout:created 2024-09-17T03:13:10.622351+0000 2024-09-17T03:13:10.620 INFO:teuthology.orchestra.run.smithi087.stdout:min_mon_release 17 (quincy) 2024-09-17T03:13:10.621 INFO:teuthology.orchestra.run.smithi087.stdout:election_strategy: 1 2024-09-17T03:13:10.621 INFO:teuthology.orchestra.run.smithi087.stdout:0: [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] mon.a 2024-09-17T03:13:10.621 INFO:teuthology.orchestra.run.smithi087.stdout:1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.b 2024-09-17T03:13:10.621 INFO:teuthology.orchestra.run.smithi087.stdout:2: [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] mon.c 2024-09-17T03:13:10.621 INFO:teuthology.orchestra.run.smithi087.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-17T03:13:10.622 DEBUG:teuthology.orchestra.run.smithi087:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-17T03:13:10.637 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 7b2553aa-e562-4da8-b175-8e91f460da55... 2024-09-17T03:13:10.638 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T03:13:10.680 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-17T03:13:10.720 INFO:teuthology.orchestra.run.smithi087.stdout:[global] 2024-09-17T03:13:10.720 INFO:teuthology.orchestra.run.smithi087.stdout: chdir = "" 2024-09-17T03:13:10.720 INFO:teuthology.orchestra.run.smithi087.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T03:13:10.720 INFO:teuthology.orchestra.run.smithi087.stdout: auth supported = cephx 2024-09-17T03:13:10.720 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.720 INFO:teuthology.orchestra.run.smithi087.stdout: filestore xattr use omap = true 2024-09-17T03:13:10.720 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.720 INFO:teuthology.orchestra.run.smithi087.stdout: mon clock drift allowed = 1.000 2024-09-17T03:13:10.720 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.721 INFO:teuthology.orchestra.run.smithi087.stdout: osd crush chooseleaf type = 0 2024-09-17T03:13:10.721 INFO:teuthology.orchestra.run.smithi087.stdout: auth debug = true 2024-09-17T03:13:10.721 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.721 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on old message = true 2024-09-17T03:13:10.721 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on bug = true 2024-09-17T03:13:10.721 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.721 INFO:teuthology.orchestra.run.smithi087.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T03:13:10.721 INFO:teuthology.orchestra.run.smithi087.stdout: mon pg warn max object skew = 0 2024-09-17T03:13:10.721 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.721 INFO:teuthology.orchestra.run.smithi087.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T03:13:10.721 INFO:teuthology.orchestra.run.smithi087.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T03:13:10.721 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.721 INFO:teuthology.orchestra.run.smithi087.stdout: osd pool default size = 2 2024-09-17T03:13:10.722 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.722 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd allow primary affinity = true 2024-09-17T03:13:10.722 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd allow pg remap = true 2024-09-17T03:13:10.722 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on legacy crush tunables = false 2024-09-17T03:13:10.722 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on crush straw calc version zero = false 2024-09-17T03:13:10.722 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on no sortbitwise = false 2024-09-17T03:13:10.722 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on osd down out interval zero = false 2024-09-17T03:13:10.722 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on too few osds = false 2024-09-17T03:13:10.722 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T03:13:10.722 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T03:13:10.722 INFO:teuthology.orchestra.run.smithi087.stdout: mon_allow_pool_size_one = true 2024-09-17T03:13:10.722 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.722 INFO:teuthology.orchestra.run.smithi087.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T03:13:10.722 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.723 INFO:teuthology.orchestra.run.smithi087.stdout: osd default data pool replay window = 5 2024-09-17T03:13:10.723 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.723 INFO:teuthology.orchestra.run.smithi087.stdout: mon allow pool delete = true 2024-09-17T03:13:10.723 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.723 INFO:teuthology.orchestra.run.smithi087.stdout: mon cluster log file level = debug 2024-09-17T03:13:10.723 INFO:teuthology.orchestra.run.smithi087.stdout: debug asserts on shutdown = true 2024-09-17T03:13:10.723 INFO:teuthology.orchestra.run.smithi087.stdout: mon health detail to clog = false 2024-09-17T03:13:10.723 INFO:teuthology.orchestra.run.smithi087.stdout: mon host = "172.21.15.87,172.21.15.149,[v2:172.21.15.149:3301,v1:172.21.15.149:6790]" 2024-09-17T03:13:10.723 INFO:teuthology.orchestra.run.smithi087.stdout: fsid = 7b2553aa-e562-4da8-b175-8e91f460da55 2024-09-17T03:13:10.723 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.723 INFO:teuthology.orchestra.run.smithi087.stdout:[osd] 2024-09-17T03:13:10.723 INFO:teuthology.orchestra.run.smithi087.stdout: osd journal size = 100 2024-09-17T03:13:10.723 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.723 INFO:teuthology.orchestra.run.smithi087.stdout: osd scrub load threshold = 5.0 2024-09-17T03:13:10.723 INFO:teuthology.orchestra.run.smithi087.stdout: osd scrub max interval = 600 2024-09-17T03:13:10.724 INFO:teuthology.orchestra.run.smithi087.stdout: osd mclock profile = high_recovery_ops 2024-09-17T03:13:10.724 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.724 INFO:teuthology.orchestra.run.smithi087.stdout: osd recover clone overlap = true 2024-09-17T03:13:10.724 INFO:teuthology.orchestra.run.smithi087.stdout: osd recovery max chunk = 1048576 2024-09-17T03:13:10.724 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.724 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug shutdown = true 2024-09-17T03:13:10.724 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug op order = true 2024-09-17T03:13:10.724 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug verify stray on activate = true 2024-09-17T03:13:10.724 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug trim objects = true 2024-09-17T03:13:10.724 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.724 INFO:teuthology.orchestra.run.smithi087.stdout: osd open classes on start = true 2024-09-17T03:13:10.724 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug pg log writeout = true 2024-09-17T03:13:10.724 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.725 INFO:teuthology.orchestra.run.smithi087.stdout: osd deep scrub update digest min age = 30 2024-09-17T03:13:10.725 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.725 INFO:teuthology.orchestra.run.smithi087.stdout: osd map max advance = 10 2024-09-17T03:13:10.725 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.725 INFO:teuthology.orchestra.run.smithi087.stdout: journal zero on create = true 2024-09-17T03:13:10.725 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.725 INFO:teuthology.orchestra.run.smithi087.stdout: filestore ondisk finisher threads = 3 2024-09-17T03:13:10.725 INFO:teuthology.orchestra.run.smithi087.stdout: filestore apply finisher threads = 3 2024-09-17T03:13:10.725 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.725 INFO:teuthology.orchestra.run.smithi087.stdout: bdev debug aio = true 2024-09-17T03:13:10.725 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug misdirected ops = true 2024-09-17T03:13:10.725 INFO:teuthology.orchestra.run.smithi087.stdout: bdev async discard = True 2024-09-17T03:13:10.725 INFO:teuthology.orchestra.run.smithi087.stdout: bdev enable discard = True 2024-09-17T03:13:10.725 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore allocator = bitmap 2024-09-17T03:13:10.726 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore block size = 96636764160 2024-09-17T03:13:10.726 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore fsck on mount = True 2024-09-17T03:13:10.726 INFO:teuthology.orchestra.run.smithi087.stdout: debug bluefs = 1/20 2024-09-17T03:13:10.726 INFO:teuthology.orchestra.run.smithi087.stdout: debug bluestore = 1/20 2024-09-17T03:13:10.726 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-09-17T03:13:10.726 INFO:teuthology.orchestra.run.smithi087.stdout: debug osd = 20 2024-09-17T03:13:10.726 INFO:teuthology.orchestra.run.smithi087.stdout: debug rocksdb = 4/10 2024-09-17T03:13:10.726 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T03:13:10.726 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd full ratio = 0.9 2024-09-17T03:13:10.726 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd nearfull ratio = 0.8 2024-09-17T03:13:10.726 INFO:teuthology.orchestra.run.smithi087.stdout: osd failsafe full ratio = 0.95 2024-09-17T03:13:10.726 INFO:teuthology.orchestra.run.smithi087.stdout: osd objectstore = bluestore 2024-09-17T03:13:10.726 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.727 INFO:teuthology.orchestra.run.smithi087.stdout:[mgr] 2024-09-17T03:13:10.727 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-09-17T03:13:10.727 INFO:teuthology.orchestra.run.smithi087.stdout: debug mgr = 20 2024-09-17T03:13:10.727 INFO:teuthology.orchestra.run.smithi087.stdout: debug mon = 20 2024-09-17T03:13:10.727 INFO:teuthology.orchestra.run.smithi087.stdout: debug auth = 20 2024-09-17T03:13:10.727 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min pgs per osd = 4 2024-09-17T03:13:10.727 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min bytes per osd = 10 2024-09-17T03:13:10.727 INFO:teuthology.orchestra.run.smithi087.stdout: mgr/telemetry/nag = false 2024-09-17T03:13:10.727 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.727 INFO:teuthology.orchestra.run.smithi087.stdout:[mon] 2024-09-17T03:13:10.728 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-09-17T03:13:10.728 INFO:teuthology.orchestra.run.smithi087.stdout: debug mon = 20 2024-09-17T03:13:10.728 INFO:teuthology.orchestra.run.smithi087.stdout: debug paxos = 20 2024-09-17T03:13:10.728 INFO:teuthology.orchestra.run.smithi087.stdout: debug auth = 20 2024-09-17T03:13:10.728 INFO:teuthology.orchestra.run.smithi087.stdout: mon data avail warn = 5 2024-09-17T03:13:10.728 INFO:teuthology.orchestra.run.smithi087.stdout: mon mgr mkfs grace = 240 2024-09-17T03:13:10.728 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min pgs per osd = 4 2024-09-17T03:13:10.728 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd reporter subtree level = osd 2024-09-17T03:13:10.728 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd prime pg temp = true 2024-09-17T03:13:10.728 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min bytes per osd = 10 2024-09-17T03:13:10.728 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.728 INFO:teuthology.orchestra.run.smithi087.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T03:13:10.728 INFO:teuthology.orchestra.run.smithi087.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T03:13:10.728 INFO:teuthology.orchestra.run.smithi087.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T03:13:10.729 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.729 INFO:teuthology.orchestra.run.smithi087.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T03:13:10.729 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T03:13:10.729 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T03:13:10.729 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.729 INFO:teuthology.orchestra.run.smithi087.stdout: # 1m isn't quite enough 2024-09-17T03:13:10.729 INFO:teuthology.orchestra.run.smithi087.stdout: mon_down_mkfs_grace = 2m 2024-09-17T03:13:10.729 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.729 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_filestore_osds = false 2024-09-17T03:13:10.729 INFO:teuthology.orchestra.run.smithi087.stdout: mon down mkfs grace = 300 2024-09-17T03:13:10.729 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:10.729 INFO:teuthology.orchestra.run.smithi087.stdout:[client] 2024-09-17T03:13:10.729 INFO:teuthology.orchestra.run.smithi087.stdout: rgw cache enabled = true 2024-09-17T03:13:10.730 INFO:teuthology.orchestra.run.smithi087.stdout: rgw enable ops log = true 2024-09-17T03:13:10.730 INFO:teuthology.orchestra.run.smithi087.stdout: rgw enable usage log = true 2024-09-17T03:13:10.730 INFO:teuthology.orchestra.run.smithi087.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T03:13:10.730 INFO:teuthology.orchestra.run.smithi087.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T03:13:10.730 INFO:teuthology.orchestra.run.smithi087.stdout: pid file = "" 2024-09-17T03:13:10.730 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.a] 2024-09-17T03:13:10.730 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.b] 2024-09-17T03:13:10.730 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.c] 2024-09-17T03:13:10.730 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.a] 2024-09-17T03:13:10.730 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.c] 2024-09-17T03:13:10.730 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.b] 2024-09-17T03:13:10.730 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.d] 2024-09-17T03:13:10.731 INFO:teuthology.orchestra.run.smithi149.stdout:[global] 2024-09-17T03:13:10.731 INFO:teuthology.orchestra.run.smithi149.stdout: chdir = "" 2024-09-17T03:13:10.731 INFO:teuthology.orchestra.run.smithi149.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-17T03:13:10.731 INFO:teuthology.orchestra.run.smithi149.stdout: auth supported = cephx 2024-09-17T03:13:10.731 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.731 INFO:teuthology.orchestra.run.smithi149.stdout: filestore xattr use omap = true 2024-09-17T03:13:10.731 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.731 INFO:teuthology.orchestra.run.smithi149.stdout: mon clock drift allowed = 1.000 2024-09-17T03:13:10.731 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: osd crush chooseleaf type = 0 2024-09-17T03:13:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: auth debug = true 2024-09-17T03:13:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: ms die on old message = true 2024-09-17T03:13:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: ms die on bug = true 2024-09-17T03:13:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-17T03:13:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: mon pg warn max object skew = 0 2024-09-17T03:13:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: # disable pg_autoscaler by default for new pools 2024-09-17T03:13:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-17T03:13:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: osd pool default size = 2 2024-09-17T03:13:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd allow primary affinity = true 2024-09-17T03:13:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd allow pg remap = true 2024-09-17T03:13:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on legacy crush tunables = false 2024-09-17T03:13:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on crush straw calc version zero = false 2024-09-17T03:13:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on no sortbitwise = false 2024-09-17T03:13:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on osd down out interval zero = false 2024-09-17T03:13:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on too few osds = false 2024-09-17T03:13:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-17T03:13:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-17T03:13:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: mon_allow_pool_size_one = true 2024-09-17T03:13:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-17T03:13:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: osd default data pool replay window = 5 2024-09-17T03:13:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: mon allow pool delete = true 2024-09-17T03:13:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: mon cluster log file level = debug 2024-09-17T03:13:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: debug asserts on shutdown = true 2024-09-17T03:13:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: mon health detail to clog = false 2024-09-17T03:13:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: mon host = "172.21.15.87,172.21.15.149,[v2:172.21.15.149:3301,v1:172.21.15.149:6790]" 2024-09-17T03:13:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: fsid = 7b2553aa-e562-4da8-b175-8e91f460da55 2024-09-17T03:13:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.734 INFO:teuthology.orchestra.run.smithi149.stdout:[osd] 2024-09-17T03:13:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: osd journal size = 100 2024-09-17T03:13:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: osd scrub load threshold = 5.0 2024-09-17T03:13:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: osd scrub max interval = 600 2024-09-17T03:13:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: osd mclock profile = high_recovery_ops 2024-09-17T03:13:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: osd recover clone overlap = true 2024-09-17T03:13:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: osd recovery max chunk = 1048576 2024-09-17T03:13:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug shutdown = true 2024-09-17T03:13:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug op order = true 2024-09-17T03:13:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug verify stray on activate = true 2024-09-17T03:13:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug trim objects = true 2024-09-17T03:13:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: osd open classes on start = true 2024-09-17T03:13:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug pg log writeout = true 2024-09-17T03:13:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: osd deep scrub update digest min age = 30 2024-09-17T03:13:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: osd map max advance = 10 2024-09-17T03:13:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: journal zero on create = true 2024-09-17T03:13:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: filestore ondisk finisher threads = 3 2024-09-17T03:13:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: filestore apply finisher threads = 3 2024-09-17T03:13:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: bdev debug aio = true 2024-09-17T03:13:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug misdirected ops = true 2024-09-17T03:13:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: bdev async discard = True 2024-09-17T03:13:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: bdev enable discard = True 2024-09-17T03:13:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: bluestore allocator = bitmap 2024-09-17T03:13:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: bluestore block size = 96636764160 2024-09-17T03:13:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: bluestore fsck on mount = True 2024-09-17T03:13:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: debug bluefs = 1/20 2024-09-17T03:13:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: debug bluestore = 1/20 2024-09-17T03:13:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-09-17T03:13:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: debug osd = 20 2024-09-17T03:13:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: debug rocksdb = 4/10 2024-09-17T03:13:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd backfillfull_ratio = 0.85 2024-09-17T03:13:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd full ratio = 0.9 2024-09-17T03:13:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd nearfull ratio = 0.8 2024-09-17T03:13:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: osd failsafe full ratio = 0.95 2024-09-17T03:13:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: osd objectstore = bluestore 2024-09-17T03:13:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.738 INFO:teuthology.orchestra.run.smithi149.stdout:[mgr] 2024-09-17T03:13:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-09-17T03:13:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: debug mgr = 20 2024-09-17T03:13:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: debug mon = 20 2024-09-17T03:13:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: debug auth = 20 2024-09-17T03:13:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min pgs per osd = 4 2024-09-17T03:13:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min bytes per osd = 10 2024-09-17T03:13:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: mgr/telemetry/nag = false 2024-09-17T03:13:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.738 INFO:teuthology.orchestra.run.smithi149.stdout:[mon] 2024-09-17T03:13:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-09-17T03:13:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: debug mon = 20 2024-09-17T03:13:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: debug paxos = 20 2024-09-17T03:13:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: debug auth = 20 2024-09-17T03:13:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: mon data avail warn = 5 2024-09-17T03:13:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: mon mgr mkfs grace = 240 2024-09-17T03:13:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min pgs per osd = 4 2024-09-17T03:13:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd reporter subtree level = osd 2024-09-17T03:13:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd prime pg temp = true 2024-09-17T03:13:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min bytes per osd = 10 2024-09-17T03:13:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-17T03:13:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: auth mon ticket ttl = 660 # 11m 2024-09-17T03:13:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: auth service ticket ttl = 240 # 4m 2024-09-17T03:13:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: # don't complain about insecure global_id in the test suite 2024-09-17T03:13:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-17T03:13:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-17T03:13:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: # 1m isn't quite enough 2024-09-17T03:13:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: mon_down_mkfs_grace = 2m 2024-09-17T03:13:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_filestore_osds = false 2024-09-17T03:13:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: mon down mkfs grace = 300 2024-09-17T03:13:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-09-17T03:13:10.740 INFO:teuthology.orchestra.run.smithi149.stdout:[client] 2024-09-17T03:13:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: rgw cache enabled = true 2024-09-17T03:13:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: rgw enable ops log = true 2024-09-17T03:13:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: rgw enable usage log = true 2024-09-17T03:13:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-17T03:13:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-17T03:13:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: pid file = "" 2024-09-17T03:13:10.740 INFO:teuthology.orchestra.run.smithi149.stdout:[mon.a] 2024-09-17T03:13:10.741 INFO:teuthology.orchestra.run.smithi149.stdout:[mon.b] 2024-09-17T03:13:10.741 INFO:teuthology.orchestra.run.smithi149.stdout:[mon.c] 2024-09-17T03:13:10.741 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.a] 2024-09-17T03:13:10.741 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.c] 2024-09-17T03:13:10.741 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.b] 2024-09-17T03:13:10.741 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.d] 2024-09-17T03:13:10.742 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-17T03:13:10.743 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-17T03:13:10.847 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-17T03:13:10.847 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:13:10.848 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-17T03:13:10.863 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:13:10.863 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-17T03:13:10.918 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi087.front.sepia.ceph.com 2024-09-17T03:13:10.919 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:13:10.919 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T03:13:10.919 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T03:13:10.993 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:13:10.993 DEBUG:teuthology.orchestra.run.smithi087:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:13:11.048 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi149.front.sepia.ceph.com 2024-09-17T03:13:11.048 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:13:11.050 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-17T03:13:11.050 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-17T03:13:11.085 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:13:11.085 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:13:11.141 INFO:tasks.ceph:Setting up mon nodes... 2024-09-17T03:13:11.141 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-17T03:13:11.141 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T03:13:11.201 INFO:teuthology.orchestra.run.smithi087.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-17T03:13:11.204 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T03:13:11.280 INFO:teuthology.orchestra.run.smithi149.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-17T03:13:11.283 INFO:tasks.ceph:Setting up mds nodes... 2024-09-17T03:13:11.284 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-17T03:13:11.344 INFO:teuthology.orchestra.run.smithi087.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-17T03:13:11.347 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-17T03:13:11.371 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-17T03:13:11.472 INFO:teuthology.orchestra.run.smithi087.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-17T03:13:11.475 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-17T03:13:11.500 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-17T03:13:11.561 INFO:teuthology.orchestra.run.smithi149.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-17T03:13:11.564 DEBUG:teuthology.orchestra.run.smithi149:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-17T03:13:11.589 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-17T03:13:11.690 INFO:teuthology.orchestra.run.smithi149.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-17T03:13:11.693 DEBUG:teuthology.orchestra.run.smithi149:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-17T03:13:11.718 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-17T03:13:11.718 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-17T03:13:11.770 INFO:teuthology.orchestra.run.smithi087.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-17T03:13:11.783 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-17T03:13:11.833 INFO:teuthology.orchestra.run.smithi087.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-17T03:13:11.846 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-17T03:13:11.898 INFO:teuthology.orchestra.run.smithi149.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-17T03:13:11.911 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-17T03:13:11.962 INFO:teuthology.orchestra.run.smithi149.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-17T03:13:11.975 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-17T03:13:11.976 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi087.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi149.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-17T03:13:11.976 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-17T03:13:12.000 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T03:13:12.000 INFO:tasks.ceph:role: osd.0 2024-09-17T03:13:12.001 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi087.front.sepia.ceph.com 2024-09-17T03:13:12.001 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T03:13:12.072 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T03:13:12.072 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T03:13:12.072 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T03:13:12.072 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T03:13:12.072 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T03:13:12.072 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2024-09-17T03:13:12.073 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T03:13:12.073 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T03:13:12.073 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T03:13:12.073 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T03:13:12.580 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2024-09-17T03:13:12.581 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2024-09-17T03:13:12.582 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-17T03:13:12.694 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-17T03:13:12.727 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-17T03:13:12.791 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T03:13:12.791 INFO:tasks.ceph:role: osd.1 2024-09-17T03:13:12.791 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi087.front.sepia.ceph.com 2024-09-17T03:13:12.791 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T03:13:12.863 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T03:13:12.864 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T03:13:12.864 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T03:13:12.864 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T03:13:12.864 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T03:13:12.864 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2024-09-17T03:13:12.864 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T03:13:12.864 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T03:13:12.864 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T03:13:12.864 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T03:13:13.312 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2024-09-17T03:13:13.314 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2024-09-17T03:13:13.315 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-17T03:13:13.387 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-17T03:13:13.456 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-17T03:13:13.522 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T03:13:13.522 INFO:tasks.ceph:role: osd.2 2024-09-17T03:13:13.522 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi087.front.sepia.ceph.com 2024-09-17T03:13:13.522 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T03:13:13.594 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T03:13:13.595 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T03:13:13.595 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T03:13:13.595 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T03:13:13.595 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T03:13:13.595 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2024-09-17T03:13:13.595 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T03:13:13.595 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T03:13:13.595 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T03:13:13.595 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T03:13:14.094 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2024-09-17T03:13:14.096 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2024-09-17T03:13:14.096 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-17T03:13:14.192 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-17T03:13:14.221 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-17T03:13:14.287 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-17T03:13:14.288 INFO:tasks.ceph:role: osd.3 2024-09-17T03:13:14.288 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi087.front.sepia.ceph.com 2024-09-17T03:13:14.288 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T03:13:14.359 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T03:13:14.359 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T03:13:14.359 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T03:13:14.360 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T03:13:14.360 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T03:13:14.360 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2024-09-17T03:13:14.360 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T03:13:14.360 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T03:13:14.360 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T03:13:14.360 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T03:13:14.886 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2024-09-17T03:13:14.887 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2024-09-17T03:13:14.888 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-17T03:13:15.037 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-17T03:13:15.072 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:13:15.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:15.174+0000 7f2d8652e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-17T03:13:15.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:15.179+0000 7f2d8652e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-17T03:13:15.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:15.179+0000 7f2d8652e640 -1 bdev(0x55a9e92c9900 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-17T03:13:15.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:15.179+0000 7f2d8652e640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-17T03:13:18.602 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-17T03:13:18.631 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:13:18.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:18.731+0000 7f9b94618640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-17T03:13:18.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:18.732+0000 7f9b94618640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-17T03:13:18.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:18.732+0000 7f9b94618640 -1 bdev(0x56164d9e9900 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-17T03:13:18.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:18.732+0000 7f9b94618640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-17T03:13:22.145 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-17T03:13:22.174 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:13:22.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:22.276+0000 7fb89b618640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-17T03:13:22.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:22.277+0000 7fb89b618640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-17T03:13:22.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:22.277+0000 7fb89b618640 -1 bdev(0x55820929b900 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-17T03:13:22.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:22.277+0000 7fb89b618640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-17T03:13:25.438 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-17T03:13:25.467 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:13:25.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:25.569+0000 7f5166620640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-17T03:13:25.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:25.569+0000 7f5166620640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-17T03:13:25.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:25.569+0000 7f5166620640 -1 bdev(0x55e1377f3900 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-17T03:13:25.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:25.569+0000 7f5166620640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-17T03:13:28.731 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-17T03:13:28.760 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-17T03:13:28.790 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T03:13:28.791 INFO:tasks.ceph:role: osd.4 2024-09-17T03:13:28.791 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi149.front.sepia.ceph.com 2024-09-17T03:13:28.791 DEBUG:teuthology.orchestra.run.smithi149:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-17T03:13:28.866 INFO:teuthology.orchestra.run.smithi149.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T03:13:28.866 INFO:teuthology.orchestra.run.smithi149.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T03:13:28.867 INFO:teuthology.orchestra.run.smithi149.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T03:13:28.867 INFO:teuthology.orchestra.run.smithi149.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T03:13:28.867 INFO:teuthology.orchestra.run.smithi149.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T03:13:28.867 INFO:teuthology.orchestra.run.smithi149.stdout: = sunit=0 swidth=0 blks 2024-09-17T03:13:28.867 INFO:teuthology.orchestra.run.smithi149.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T03:13:28.867 INFO:teuthology.orchestra.run.smithi149.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T03:13:28.867 INFO:teuthology.orchestra.run.smithi149.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T03:13:28.867 INFO:teuthology.orchestra.run.smithi149.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T03:13:29.385 INFO:teuthology.orchestra.run.smithi149.stdout:Discarding blocks...Done. 2024-09-17T03:13:29.387 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi149.front.sepia.ceph.com -o noatime 2024-09-17T03:13:29.387 DEBUG:teuthology.orchestra.run.smithi149:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-17T03:13:29.513 DEBUG:teuthology.orchestra.run.smithi149:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-17T03:13:29.543 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-17T03:13:29.609 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T03:13:29.609 INFO:tasks.ceph:role: osd.5 2024-09-17T03:13:29.609 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi149.front.sepia.ceph.com 2024-09-17T03:13:29.610 DEBUG:teuthology.orchestra.run.smithi149:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-17T03:13:29.682 INFO:teuthology.orchestra.run.smithi149.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T03:13:29.682 INFO:teuthology.orchestra.run.smithi149.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T03:13:29.682 INFO:teuthology.orchestra.run.smithi149.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T03:13:29.683 INFO:teuthology.orchestra.run.smithi149.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T03:13:29.683 INFO:teuthology.orchestra.run.smithi149.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T03:13:29.683 INFO:teuthology.orchestra.run.smithi149.stdout: = sunit=0 swidth=0 blks 2024-09-17T03:13:29.683 INFO:teuthology.orchestra.run.smithi149.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T03:13:29.683 INFO:teuthology.orchestra.run.smithi149.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T03:13:29.683 INFO:teuthology.orchestra.run.smithi149.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T03:13:29.683 INFO:teuthology.orchestra.run.smithi149.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T03:13:30.141 INFO:teuthology.orchestra.run.smithi149.stdout:Discarding blocks...Done. 2024-09-17T03:13:30.143 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi149.front.sepia.ceph.com -o noatime 2024-09-17T03:13:30.143 DEBUG:teuthology.orchestra.run.smithi149:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-17T03:13:30.234 DEBUG:teuthology.orchestra.run.smithi149:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-17T03:13:30.262 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-17T03:13:30.327 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T03:13:30.327 INFO:tasks.ceph:role: osd.6 2024-09-17T03:13:30.327 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi149.front.sepia.ceph.com 2024-09-17T03:13:30.327 DEBUG:teuthology.orchestra.run.smithi149:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-17T03:13:30.399 INFO:teuthology.orchestra.run.smithi149.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T03:13:30.399 INFO:teuthology.orchestra.run.smithi149.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T03:13:30.399 INFO:teuthology.orchestra.run.smithi149.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T03:13:30.399 INFO:teuthology.orchestra.run.smithi149.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T03:13:30.399 INFO:teuthology.orchestra.run.smithi149.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T03:13:30.399 INFO:teuthology.orchestra.run.smithi149.stdout: = sunit=0 swidth=0 blks 2024-09-17T03:13:30.399 INFO:teuthology.orchestra.run.smithi149.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T03:13:30.399 INFO:teuthology.orchestra.run.smithi149.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T03:13:30.399 INFO:teuthology.orchestra.run.smithi149.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T03:13:30.399 INFO:teuthology.orchestra.run.smithi149.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T03:13:30.886 INFO:teuthology.orchestra.run.smithi149.stdout:Discarding blocks...Done. 2024-09-17T03:13:30.888 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi149.front.sepia.ceph.com -o noatime 2024-09-17T03:13:30.888 DEBUG:teuthology.orchestra.run.smithi149:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-17T03:13:31.016 DEBUG:teuthology.orchestra.run.smithi149:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-17T03:13:31.045 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-17T03:13:31.110 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-17T03:13:31.110 INFO:tasks.ceph:role: osd.7 2024-09-17T03:13:31.110 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi149.front.sepia.ceph.com 2024-09-17T03:13:31.110 DEBUG:teuthology.orchestra.run.smithi149:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-17T03:13:31.182 INFO:teuthology.orchestra.run.smithi149.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-17T03:13:31.183 INFO:teuthology.orchestra.run.smithi149.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-17T03:13:31.183 INFO:teuthology.orchestra.run.smithi149.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-17T03:13:31.183 INFO:teuthology.orchestra.run.smithi149.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-17T03:13:31.183 INFO:teuthology.orchestra.run.smithi149.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-17T03:13:31.183 INFO:teuthology.orchestra.run.smithi149.stdout: = sunit=0 swidth=0 blks 2024-09-17T03:13:31.183 INFO:teuthology.orchestra.run.smithi149.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-17T03:13:31.183 INFO:teuthology.orchestra.run.smithi149.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-17T03:13:31.183 INFO:teuthology.orchestra.run.smithi149.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-17T03:13:31.183 INFO:teuthology.orchestra.run.smithi149.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-17T03:13:31.713 INFO:teuthology.orchestra.run.smithi149.stdout:Discarding blocks...Done. 2024-09-17T03:13:31.715 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi149.front.sepia.ceph.com -o noatime 2024-09-17T03:13:31.715 DEBUG:teuthology.orchestra.run.smithi149:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-17T03:13:31.841 DEBUG:teuthology.orchestra.run.smithi149:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-17T03:13:31.872 DEBUG:teuthology.orchestra.run.smithi149:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:13:31.988 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:31.989+0000 7fa7b4dbf640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-17T03:13:31.989 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:31.990+0000 7fa7b4dbf640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-17T03:13:31.989 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:31.990+0000 7fa7b4dbf640 -1 bdev(0x55a7d9c77900 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-17T03:13:31.989 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:31.990+0000 7fa7b4dbf640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-17T03:13:35.646 DEBUG:teuthology.orchestra.run.smithi149:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-17T03:13:35.674 DEBUG:teuthology.orchestra.run.smithi149:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:13:35.774 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:35.774+0000 7f9f921dd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-17T03:13:35.774 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:35.775+0000 7f9f921dd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-17T03:13:35.774 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:35.775+0000 7f9f921dd640 -1 bdev(0x555666747900 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-17T03:13:35.774 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:35.775+0000 7f9f921dd640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-17T03:13:38.759 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:13:38.762 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:13:39.426 DEBUG:teuthology.orchestra.run.smithi149:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-17T03:13:39.454 DEBUG:teuthology.orchestra.run.smithi149:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:13:39.555 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:39.555+0000 7ff00c830640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-17T03:13:39.555 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:39.555+0000 7ff00c830640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-17T03:13:39.555 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:39.555+0000 7ff00c830640 -1 bdev(0x55cff1a97900 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-17T03:13:39.555 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:39.556+0000 7ff00c830640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-17T03:13:42.716 DEBUG:teuthology.orchestra.run.smithi149:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-17T03:13:42.743 DEBUG:teuthology.orchestra.run.smithi149:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:13:42.841 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:42.841+0000 7f66a4044640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-17T03:13:42.842 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:42.842+0000 7f66a4044640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-17T03:13:42.842 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:42.842+0000 7f66a4044640 -1 bdev(0x55e30199f900 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-17T03:13:42.842 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:42.842+0000 7f66a4044640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-17T03:13:45.750 DEBUG:teuthology.orchestra.run.smithi149:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-17T03:13:45.777 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-17T03:13:45.778 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:13:45.778 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-17T03:13:45.807 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:13:45.808 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-17T03:13:45.875 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:13:45.875 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-17T03:13:45.942 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:13:45.942 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-17T03:13:46.009 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:13:46.009 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-17T03:13:46.075 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:13:46.075 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-17T03:13:46.141 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:13:46.141 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-17T03:13:46.206 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:13:46.206 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-17T03:13:46.237 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:13:46.237 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-17T03:13:46.306 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:13:46.306 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-17T03:13:46.374 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:13:46.374 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-17T03:13:46.441 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:13:46.441 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-17T03:13:46.508 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:13:46.508 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-17T03:13:46.575 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:13:46.575 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-17T03:13:46.640 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:13:46.641 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-17T03:13:46.657 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:13:46.657 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-17T03:13:46.715 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:13:46.715 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-17T03:13:46.731 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:13:46.731 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-17T03:13:46.786 INFO:tasks.ceph:Adding keys to all mons... 2024-09-17T03:13:46.787 DEBUG:teuthology.orchestra.run.smithi087:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T03:13:46.789 DEBUG:teuthology.orchestra.run.smithi149:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-17T03:13:46.832 INFO:teuthology.orchestra.run.smithi087.stdout:[mgr.y] 2024-09-17T03:13:46.833 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDH8+hmsBkdDBAA4bwvaWSWANblx5JF75wTQQ== 2024-09-17T03:13:46.833 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.a] 2024-09-17T03:13:46.833 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDH8+hmLoKqFBAAl7DDFJqmrUwbDOL3J8wysA== 2024-09-17T03:13:46.833 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.c] 2024-09-17T03:13:46.833 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDH8+hm+QtNHBAANIlkESYiJ80tcoyvu5KZLA== 2024-09-17T03:13:46.833 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.0] 2024-09-17T03:13:46.833 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDL8+hm7Rx0ChAA8g5yLI7cCaX55jkLjNkzQA== 2024-09-17T03:13:46.833 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.1] 2024-09-17T03:13:46.833 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDO8+hmQzq8KxAA5LsvmZb7P2FIPXd9wc23Ig== 2024-09-17T03:13:46.833 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.2] 2024-09-17T03:13:46.833 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDS8+hm0jabEBAAYJIY3OgCoJ4EcYBJr89xSQ== 2024-09-17T03:13:46.834 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.3] 2024-09-17T03:13:46.834 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDV8+hmSrsMIhAAWW1HvxXttwHNUJT5NxsPYA== 2024-09-17T03:13:46.834 INFO:teuthology.orchestra.run.smithi087.stdout:[mgr.x] 2024-09-17T03:13:46.834 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDH8+hmGA/bEBAA7l5te6R2YeXND6keGQpVSw== 2024-09-17T03:13:46.834 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.b] 2024-09-17T03:13:46.834 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDH8+hmOziYIRAA2hVbfPg5KtY+dF/o1HzAFA== 2024-09-17T03:13:46.834 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.d] 2024-09-17T03:13:46.834 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDH8+hmRUxHKRAARWfxi+PE89hD4fMhCdbnxQ== 2024-09-17T03:13:46.834 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.4] 2024-09-17T03:13:46.835 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDb8+hmg54IOxAAKQ8plXDnZxp7QlGZcNOVXg== 2024-09-17T03:13:46.835 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.5] 2024-09-17T03:13:46.835 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDf8+hmUT1GLhAA6NQyWAThxgYjnf2X+PFE0A== 2024-09-17T03:13:46.835 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.6] 2024-09-17T03:13:46.835 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDj8+hmjDo2IRAA6UyXPCnTHBilelz+S6IWmQ== 2024-09-17T03:13:46.835 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.7] 2024-09-17T03:13:46.835 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDm8+hmJKtHMhAA56cm6L9B9GLL48bRfjBnLg== 2024-09-17T03:13:46.835 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-09-17T03:13:46.835 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDH8+hmLskNLhAAd2M5ynGP8W6Pim11F18wYA== 2024-09-17T03:13:46.835 INFO:teuthology.orchestra.run.smithi087.stdout:[client.2] 2024-09-17T03:13:46.835 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDH8+hmqYfNMRAAzS0etgXaryNA6diH8MM4vQ== 2024-09-17T03:13:46.835 INFO:teuthology.orchestra.run.smithi087.stdout:[client.1] 2024-09-17T03:13:46.835 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDH8+hm+aGqNRAAUG+G7uzrYQ90DHV08QvWKQ== 2024-09-17T03:13:46.836 INFO:teuthology.orchestra.run.smithi087.stdout:[client.3] 2024-09-17T03:13:46.836 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDH8+hmKQl7ORAAV04v5s/X/fn24bp8kLIRsA== 2024-09-17T03:13:46.850 INFO:teuthology.orchestra.run.smithi149.stdout:[mgr.y] 2024-09-17T03:13:46.850 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDH8+hmsBkdDBAA4bwvaWSWANblx5JF75wTQQ== 2024-09-17T03:13:46.850 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.a] 2024-09-17T03:13:46.850 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDH8+hmLoKqFBAAl7DDFJqmrUwbDOL3J8wysA== 2024-09-17T03:13:46.850 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.c] 2024-09-17T03:13:46.850 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDH8+hm+QtNHBAANIlkESYiJ80tcoyvu5KZLA== 2024-09-17T03:13:46.850 INFO:teuthology.orchestra.run.smithi149.stdout:[osd.0] 2024-09-17T03:13:46.850 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDL8+hm7Rx0ChAA8g5yLI7cCaX55jkLjNkzQA== 2024-09-17T03:13:46.850 INFO:teuthology.orchestra.run.smithi149.stdout:[osd.1] 2024-09-17T03:13:46.850 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDO8+hmQzq8KxAA5LsvmZb7P2FIPXd9wc23Ig== 2024-09-17T03:13:46.850 INFO:teuthology.orchestra.run.smithi149.stdout:[osd.2] 2024-09-17T03:13:46.850 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDS8+hm0jabEBAAYJIY3OgCoJ4EcYBJr89xSQ== 2024-09-17T03:13:46.850 INFO:teuthology.orchestra.run.smithi149.stdout:[osd.3] 2024-09-17T03:13:46.850 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDV8+hmSrsMIhAAWW1HvxXttwHNUJT5NxsPYA== 2024-09-17T03:13:46.851 INFO:teuthology.orchestra.run.smithi149.stdout:[mgr.x] 2024-09-17T03:13:46.851 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDH8+hmGA/bEBAA7l5te6R2YeXND6keGQpVSw== 2024-09-17T03:13:46.851 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.b] 2024-09-17T03:13:46.851 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDH8+hmOziYIRAA2hVbfPg5KtY+dF/o1HzAFA== 2024-09-17T03:13:46.851 INFO:teuthology.orchestra.run.smithi149.stdout:[mds.d] 2024-09-17T03:13:46.851 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDH8+hmRUxHKRAARWfxi+PE89hD4fMhCdbnxQ== 2024-09-17T03:13:46.851 INFO:teuthology.orchestra.run.smithi149.stdout:[osd.4] 2024-09-17T03:13:46.851 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDb8+hmg54IOxAAKQ8plXDnZxp7QlGZcNOVXg== 2024-09-17T03:13:46.851 INFO:teuthology.orchestra.run.smithi149.stdout:[osd.5] 2024-09-17T03:13:46.851 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDf8+hmUT1GLhAA6NQyWAThxgYjnf2X+PFE0A== 2024-09-17T03:13:46.851 INFO:teuthology.orchestra.run.smithi149.stdout:[osd.6] 2024-09-17T03:13:46.851 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDj8+hmjDo2IRAA6UyXPCnTHBilelz+S6IWmQ== 2024-09-17T03:13:46.851 INFO:teuthology.orchestra.run.smithi149.stdout:[osd.7] 2024-09-17T03:13:46.851 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDm8+hmJKtHMhAA56cm6L9B9GLL48bRfjBnLg== 2024-09-17T03:13:46.851 INFO:teuthology.orchestra.run.smithi149.stdout:[client.0] 2024-09-17T03:13:46.852 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDH8+hmLskNLhAAd2M5ynGP8W6Pim11F18wYA== 2024-09-17T03:13:46.852 INFO:teuthology.orchestra.run.smithi149.stdout:[client.2] 2024-09-17T03:13:46.852 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDH8+hmqYfNMRAAzS0etgXaryNA6diH8MM4vQ== 2024-09-17T03:13:46.852 INFO:teuthology.orchestra.run.smithi149.stdout:[client.1] 2024-09-17T03:13:46.852 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDH8+hm+aGqNRAAUG+G7uzrYQ90DHV08QvWKQ== 2024-09-17T03:13:46.852 INFO:teuthology.orchestra.run.smithi149.stdout:[client.3] 2024-09-17T03:13:46.852 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDH8+hmKQl7ORAAV04v5s/X/fn24bp8kLIRsA== 2024-09-17T03:13:46.853 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T03:13:46.875 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T03:13:46.956 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T03:13:46.959 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T03:13:47.023 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T03:13:47.026 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T03:13:47.090 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:13:47.093 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:13:47.154 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:13:47.156 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:13:47.218 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:13:47.220 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:13:47.281 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:13:47.284 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:13:47.345 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T03:13:47.347 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-17T03:13:47.409 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T03:13:47.411 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T03:13:47.476 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T03:13:47.478 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-17T03:13:47.543 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:13:47.546 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:13:47.608 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:13:47.610 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:13:47.672 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:13:47.674 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:13:47.736 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:13:47.738 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-17T03:13:47.800 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T03:13:47.802 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T03:13:47.867 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T03:13:47.870 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T03:13:47.935 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T03:13:47.937 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T03:13:48.002 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T03:13:48.005 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-17T03:13:48.070 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-17T03:13:48.070 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-17T03:13:48.095 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T03:13:48.202 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-17T03:13:48.226 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-17T03:13:48.251 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T03:13:48.359 DEBUG:teuthology.orchestra.run.smithi149:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-17T03:13:48.384 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-17T03:13:48.449 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-17T03:13:48.556 DEBUG:teuthology.orchestra.run.smithi149:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-17T03:13:48.581 DEBUG:teuthology.orchestra.run.smithi087:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:13:48.583 DEBUG:teuthology.orchestra.run.smithi149:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-17T03:13:48.637 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-17T03:13:48.637 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-17T03:13:48.637 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-17T03:13:48.639 INFO:tasks.ceph.mon.a:Started 2024-09-17T03:13:48.639 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-17T03:13:48.639 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-17T03:13:48.679 INFO:tasks.ceph.mon.b:Started 2024-09-17T03:13:48.680 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-17T03:13:48.680 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-17T03:13:48.682 INFO:tasks.ceph.mon.c:Started 2024-09-17T03:13:48.682 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-17T03:13:48.682 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-17T03:13:48.682 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-17T03:13:48.684 INFO:tasks.ceph.mgr.y:Started 2024-09-17T03:13:48.684 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-17T03:13:48.685 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-17T03:13:48.686 INFO:tasks.ceph.mgr.x:Started 2024-09-17T03:13:48.687 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-17T03:13:48.687 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-17T03:13:48.689 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-17T03:13:48.689 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-17T03:13:48.731 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-17T03:13:48.731 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-17T03:13:48.774 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-17T03:13:48.774 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-17T03:13:48.817 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-17T03:13:48.817 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-17T03:13:48.860 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-17T03:13:48.861 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-17T03:13:48.906 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-17T03:13:48.906 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-17T03:13:48.949 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-17T03:13:48.949 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-17T03:13:48.993 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-17T03:13:48.993 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-17T03:13:49.036 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-17T03:13:49.036 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-17T03:13:49.081 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-17T03:13:49.081 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-17T03:13:49.127 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-17T03:13:49.127 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-17T03:13:49.170 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-17T03:13:49.170 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-17T03:13:49.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.197+0000 7f22cb7fe640 1 -- 172.21.15.87:0/1781433136 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f22dc044070 con 0x7f22e4071ba0 2024-09-17T03:13:49.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.197+0000 7f22cb7fe640 1 -- 172.21.15.87:0/1781433136 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f22dc03d430 con 0x7f22e4071ba0 2024-09-17T03:13:49.198 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-17T03:13:49.198 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-17T03:13:49.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.204+0000 7fd003f58640 1 -- 172.21.15.87:0/2177990303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcffc153340 msgr2=0x7fcffc1f99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.204+0000 7fd003f58640 1 --2- 172.21.15.87:0/2177990303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcffc153340 0x7fcffc1f99d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcfec009890 tx=0x7fcfec009d50 comp rx=0 tx=0).stop 2024-09-17T03:13:49.211 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-17T03:13:49.211 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-17T03:13:49.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.205+0000 7fd003f58640 1 -- 172.21.15.87:0/2177990303 shutdown_connections 2024-09-17T03:13:49.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.205+0000 7fd003f58640 1 --2- 172.21.15.87:0/2177990303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcffc153750 0x7fcffc15d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.205+0000 7fd003f58640 1 --2- 172.21.15.87:0/2177990303 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcffc153340 0x7fcffc1f99d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.205+0000 7fd003f58640 1 --2- 172.21.15.87:0/2177990303 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcffc0a9490 0x7fcffc1fd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.205+0000 7fd003f58640 1 -- 172.21.15.87:0/2177990303 >> 172.21.15.87:0/2177990303 conn(0x7fcffc06d440 msgr2=0x7fcffc15b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.210+0000 7f632fc38640 1 -- 172.21.15.87:0/3428467487 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f6304005310 con 0x7f63281522e0 2024-09-17T03:13:49.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.208+0000 7fd003f58640 1 -- 172.21.15.87:0/2177990303 shutdown_connections 2024-09-17T03:13:49.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.208+0000 7fd003f58640 1 -- 172.21.15.87:0/2177990303 wait complete. 2024-09-17T03:13:49.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.207+0000 7f22c97fa640 1 -- 172.21.15.87:0/1781433136 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22c4005290 con 0x7f22e4071ba0 2024-09-17T03:13:49.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.212+0000 7f22cb7fe640 1 -- 172.21.15.87:0/1781433136 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f22dc032730 con 0x7f22e4071ba0 2024-09-17T03:13:49.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.216+0000 7f631e7fc640 1 -- 172.21.15.87:0/3428467487 <== mon.0 v2:172.21.15.87:3300/0 11 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7f6318025520 con 0x7f63281522e0 2024-09-17T03:13:49.217 INFO:tasks.ceph:setting config [mgr] client_mount_timeout = 30 2024-09-17T03:13:49.217 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config set mgr client_mount_timeout 30 2024-09-17T03:13:49.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.217+0000 7f631e7fc640 1 -- 172.21.15.87:0/3428467487 <== mon.0 v2:172.21.15.87:3300/0 12 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6318025780 con 0x7f63281522e0 2024-09-17T03:13:49.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.217+0000 7fcd0e7fc640 1 -- 172.21.15.87:0/1806398154 <== mon.0 v2:172.21.15.87:3300/0 11 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd1003c390 con 0x7fcd18070c40 2024-09-17T03:13:49.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.220+0000 7f632fc38640 1 -- 172.21.15.87:0/3428467487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63281522e0 msgr2=0x7f632815b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.220+0000 7f632fc38640 1 --2- 172.21.15.87:0/3428467487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63281522e0 0x7f632815b240 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6318003f80 tx=0x7f63180018e0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.224 INFO:tasks.ceph:setting config [mgr] debug_client = 20 2024-09-17T03:13:49.224 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config set mgr debug_client 20 2024-09-17T03:13:49.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.220+0000 7f632fc38640 1 -- 172.21.15.87:0/3428467487 shutdown_connections 2024-09-17T03:13:49.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.220+0000 7f632fc38640 1 --2- 172.21.15.87:0/3428467487 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63281526f0 0x7f632815b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.220+0000 7f632fc38640 1 --2- 172.21.15.87:0/3428467487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63281522e0 0x7f632815b240 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.220+0000 7f632fc38640 1 --2- 172.21.15.87:0/3428467487 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6314005150 0x7f63281f4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.220+0000 7f632fc38640 1 -- 172.21.15.87:0/3428467487 >> 172.21.15.87:0/3428467487 conn(0x7f632806d440 msgr2=0x7f6328155250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.221+0000 7f632fc38640 1 -- 172.21.15.87:0/3428467487 shutdown_connections 2024-09-17T03:13:49.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.221+0000 7f632fc38640 1 -- 172.21.15.87:0/3428467487 wait complete. 2024-09-17T03:13:49.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.229+0000 7f22cb7fe640 1 -- 172.21.15.87:0/1781433136 <== mon.0 v2:172.21.15.87:3300/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22dc046450 con 0x7f22e4071ba0 2024-09-17T03:13:49.229 INFO:tasks.ceph:setting config [mgr] debug_mgr = 20 2024-09-17T03:13:49.229 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config set mgr debug_mgr 20 2024-09-17T03:13:49.259 INFO:tasks.ceph:setting config [mgr] debug_ms = 1 2024-09-17T03:13:49.260 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config set mgr debug_ms 1 2024-09-17T03:13:49.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.261+0000 7f7902d8a640 1 -- 172.21.15.87:0/3998069513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78fc158780 msgr2=0x7f78fc0af700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.261+0000 7f7902d8a640 1 --2- 172.21.15.87:0/3998069513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78fc158780 0x7f78fc0af700 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f78f4005fd0 tx=0x7f78f402a8d0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.261+0000 7f7902d8a640 1 -- 172.21.15.87:0/3998069513 shutdown_connections 2024-09-17T03:13:49.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.261+0000 7f7902d8a640 1 --2- 172.21.15.87:0/3998069513 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f78fc0afc40 0x7f78fc0b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.261+0000 7f7902d8a640 1 --2- 172.21.15.87:0/3998069513 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78fc158780 0x7f78fc0af700 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.261+0000 7f7902d8a640 1 -- 172.21.15.87:0/3998069513 >> 172.21.15.87:0/3998069513 conn(0x7f78fc06d440 msgr2=0x7f78fc0b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.263+0000 7f7902d8a640 1 -- 172.21.15.87:0/3998069513 shutdown_connections 2024-09-17T03:13:49.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.263+0000 7f7902d8a640 1 -- 172.21.15.87:0/3998069513 wait complete. 2024-09-17T03:13:49.266 INFO:tasks.ceph:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-17T03:13:49.266 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config set mgr mon_warn_on_pool_no_app False 2024-09-17T03:13:49.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.266+0000 7f7902d8a640 1 Processor -- start 2024-09-17T03:13:49.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.267+0000 7f7902d8a640 1 -- start start 2024-09-17T03:13:49.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.269+0000 7f7902d8a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78fc0afc40 0x7f78fc1f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.269+0000 7f7902d8a640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f78fc158780 0x7f78fc1534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.269+0000 7f7902d8a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78fc15f4b0 0x7f78fc0cf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.269+0000 7f7902d8a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f78fc0b5560 con 0x7f78fc0afc40 2024-09-17T03:13:49.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.269+0000 7f7902d8a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f78fc0b53e0 con 0x7f78fc15f4b0 2024-09-17T03:13:49.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.269+0000 7f7902d8a640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f78fc15b240 con 0x7f78fc158780 2024-09-17T03:13:49.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.270+0000 7f7900b26640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78fc0afc40 0x7f78fc1f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.270+0000 7f7900b26640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78fc0afc40 0x7f78fc1f25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40828/0 (socket says 172.21.15.87:40828) 2024-09-17T03:13:49.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.270+0000 7f7900b26640 1 -- 172.21.15.87:0/2565007635 learned_addr learned my addr 172.21.15.87:0/2565007635 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:13:49.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.270+0000 7f78fbfff640 1 --2- 172.21.15.87:0/2565007635 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f78fc158780 0x7f78fc1534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.270+0000 7f7900b26640 1 -- 172.21.15.87:0/2565007635 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f78fc158780 msgr2=0x7f78fc1534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.270+0000 7f7900b26640 1 --2- 172.21.15.87:0/2565007635 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f78fc158780 0x7f78fc1534a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.270+0000 7f7900b26640 1 -- 172.21.15.87:0/2565007635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78fc15f4b0 msgr2=0x7f78fc0cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.270+0000 7f7901327640 1 --2- 172.21.15.87:0/2565007635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78fc15f4b0 0x7f78fc0cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.270+0000 7f7900b26640 1 --2- 172.21.15.87:0/2565007635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78fc15f4b0 0x7f78fc0cf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.270+0000 7f7900b26640 1 -- 172.21.15.87:0/2565007635 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78fc1f2b10 con 0x7f78fc0afc40 2024-09-17T03:13:49.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.271+0000 7f7900b26640 1 --2- 172.21.15.87:0/2565007635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78fc0afc40 0x7f78fc1f25d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f78ec00c750 tx=0x7f78ec00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:49.274 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-17T03:13:49.274 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-17T03:13:49.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.271+0000 7f78f97fa640 1 -- 172.21.15.87:0/2565007635 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ec005070 con 0x7f78fc0afc40 2024-09-17T03:13:49.281 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-17T03:13:49.281 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-17T03:13:49.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.271+0000 7f78f97fa640 1 -- 172.21.15.87:0/2565007635 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78ec005230 con 0x7f78fc0afc40 2024-09-17T03:13:49.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.271+0000 7f78f97fa640 1 -- 172.21.15.87:0/2565007635 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ec00a640 con 0x7f78fc0afc40 2024-09-17T03:13:49.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.275+0000 7f7902d8a640 1 -- 172.21.15.87:0/2565007635 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78fc15dff0 con 0x7f78fc0afc40 2024-09-17T03:13:49.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.275+0000 7f7902d8a640 1 -- 172.21.15.87:0/2565007635 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78fc1f1dd0 con 0x7f78fc0afc40 2024-09-17T03:13:49.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.282+0000 7f78f97fa640 1 -- 172.21.15.87:0/2565007635 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f78ec00ab70 con 0x7f78fc0afc40 2024-09-17T03:13:49.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.282+0000 7f78f97fa640 1 -- 172.21.15.87:0/2565007635 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f78ec0125d0 con 0x7f78fc0afc40 2024-09-17T03:13:49.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.282+0000 7f7902d8a640 1 -- 172.21.15.87:0/2565007635 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78fc1f1bf0 con 0x7f78fc0afc40 2024-09-17T03:13:49.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.305+0000 7f78f97fa640 1 -- 172.21.15.87:0/2565007635 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f78fc1f1bf0 con 0x7f78fc0afc40 2024-09-17T03:13:49.307 DEBUG:tasks.ceph:set 22 configs 2024-09-17T03:13:49.307 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi087') 2024-09-17T03:13:49.307 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi087') 2024-09-17T03:13:49.307 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi087') 2024-09-17T03:13:49.308 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi087') 2024-09-17T03:13:49.308 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi087') 2024-09-17T03:13:49.308 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi087') 2024-09-17T03:13:49.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.310+0000 7f1509cde640 1 -- 172.21.15.87:0/248965171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15040cee70 msgr2=0x7f15040cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.310+0000 7f1509cde640 1 --2- 172.21.15.87:0/248965171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15040cee70 0x7f15040cf280 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f150406fdd0 tx=0x7f14fc02edb0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.310+0000 7f1509cde640 1 -- 172.21.15.87:0/248965171 shutdown_connections 2024-09-17T03:13:49.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.310+0000 7f1509cde640 1 --2- 172.21.15.87:0/248965171 >> v2:172.21.15.87:3300/0 conn(0x7f150416b6d0 0x7f150416bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.310+0000 7f1509cde640 1 --2- 172.21.15.87:0/248965171 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15040cee70 0x7f15040cf280 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.310+0000 7f1509cde640 1 --2- 172.21.15.87:0/248965171 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1504073540 0x7f15040af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.310+0000 7f1509cde640 1 -- 172.21.15.87:0/248965171 >> 172.21.15.87:0/248965171 conn(0x7f15040b1b60 msgr2=0x7f15040b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.311+0000 7f1509cde640 1 -- 172.21.15.87:0/248965171 shutdown_connections 2024-09-17T03:13:49.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.311+0000 7f1509cde640 1 -- 172.21.15.87:0/248965171 wait complete. 2024-09-17T03:13:49.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.314+0000 7f1509cde640 1 Processor -- start 2024-09-17T03:13:49.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.314+0000 7f1509cde640 1 -- start start 2024-09-17T03:13:49.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.316+0000 7f1509cde640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1504073540 0x7f15041f3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.317+0000 7f1509cde640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f150416b6d0 0x7f15041f89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.318+0000 7f15037fe640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1504073540 0x7f15041f3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.318+0000 7f15037fe640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1504073540 0x7f15041f3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.87:53664/0 (socket says 172.21.15.87:53664) 2024-09-17T03:13:49.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.322+0000 7f1502ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f150416b6d0 0x7f15041f89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.322+0000 7f1502ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f150416b6d0 0x7f15041f89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40840/0 (socket says 172.21.15.87:40840) 2024-09-17T03:13:49.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.322+0000 7f15037fe640 1 -- 172.21.15.87:0/1059120084 learned_addr learned my addr 172.21.15.87:0/1059120084 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:13:49.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.322+0000 7f1509cde640 1 --2- 172.21.15.87:0/1059120084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15041f7510 0x7f15041f7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.322+0000 7f1509cde640 1 -- 172.21.15.87:0/1059120084 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f150408bd30 con 0x7f150416b6d0 2024-09-17T03:13:49.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.322+0000 7f1509cde640 1 -- 172.21.15.87:0/1059120084 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f150408bbb0 con 0x7f15041f7510 2024-09-17T03:13:49.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.322+0000 7f1509cde640 1 -- 172.21.15.87:0/1059120084 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f150416c020 con 0x7f1504073540 2024-09-17T03:13:49.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.324+0000 7f1503fff640 1 --2- 172.21.15.87:0/1059120084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15041f7510 0x7f15041f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.324+0000 7f1502ffd640 1 -- 172.21.15.87:0/1059120084 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1504073540 msgr2=0x7f15041f3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.324+0000 7f1502ffd640 1 --2- 172.21.15.87:0/1059120084 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1504073540 0x7f15041f3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.324+0000 7f1502ffd640 1 -- 172.21.15.87:0/1059120084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15041f7510 msgr2=0x7f15041f7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.324+0000 7f1502ffd640 1 --2- 172.21.15.87:0/1059120084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15041f7510 0x7f15041f7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.324+0000 7f1502ffd640 1 -- 172.21.15.87:0/1059120084 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15041f5f40 con 0x7f150416b6d0 2024-09-17T03:13:49.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.324+0000 7f1502ffd640 1 --2- 172.21.15.87:0/1059120084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f150416b6d0 0x7f15041f89e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f14fc036000 tx=0x7f14fc030b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:49.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.325+0000 7f14ebfff640 1 -- 172.21.15.87:0/1059120084 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc03c070 con 0x7f150416b6d0 2024-09-17T03:13:49.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.325+0000 7f14ebfff640 1 -- 172.21.15.87:0/1059120084 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14fc0103d0 con 0x7f150416b6d0 2024-09-17T03:13:49.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.326+0000 7f1509cde640 1 -- 172.21.15.87:0/1059120084 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15041f6470 con 0x7f150416b6d0 2024-09-17T03:13:49.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.326+0000 7f1509cde640 1 -- 172.21.15.87:0/1059120084 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15041f6a30 con 0x7f150416b6d0 2024-09-17T03:13:49.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.328+0000 7f14ebfff640 1 -- 172.21.15.87:0/1059120084 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14fc037aa0 con 0x7f150416b6d0 2024-09-17T03:13:49.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.330+0000 7fccfbfff640 1 -- 172.21.15.87:0/1806398154 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7fccf40052f0 con 0x7fcd18070c40 2024-09-17T03:13:49.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.330+0000 7f14e9ffb640 1 -- 172.21.15.87:0/1059120084 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14e4005290 con 0x7f150416b6d0 2024-09-17T03:13:49.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.334+0000 7f14ebfff640 1 -- 172.21.15.87:0/1059120084 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f14fc047060 con 0x7f150416b6d0 2024-09-17T03:13:49.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.334+0000 7f14ebfff640 1 -- 172.21.15.87:0/1059120084 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f14fc049600 con 0x7f150416b6d0 2024-09-17T03:13:49.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.345+0000 7f14ebfff640 1 -- 172.21.15.87:0/1059120084 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f14fc04e020 con 0x7f150416b6d0 2024-09-17T03:13:49.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.348+0000 7fcd0e7fc640 1 -- 172.21.15.87:0/1806398154 <== mon.0 v2:172.21.15.87:3300/0 12 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v7)=0 v7) ==== 108+0+0 (secure 0 0 0) 0x7fcd18146410 con 0x7fcd18070c40 2024-09-17T03:13:49.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.349+0000 7fcd1fcc4640 1 -- 172.21.15.87:0/1806398154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd18070c40 msgr2=0x7fcd180ae600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.349+0000 7fcd1fcc4640 1 --2- 172.21.15.87:0/1806398154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd18070c40 0x7fcd180ae600 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fcd10006010 tx=0x7fcd10004a40 comp rx=0 tx=0).stop 2024-09-17T03:13:49.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.350+0000 7fcd1fcc4640 1 -- 172.21.15.87:0/1806398154 shutdown_connections 2024-09-17T03:13:49.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.350+0000 7fcd1fcc4640 1 --2- 172.21.15.87:0/1806398154 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcd180af080 0x7fcd181f9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.350+0000 7fcd1fcc4640 1 --2- 172.21.15.87:0/1806398154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcd180b38c0 0x7fcd180aeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.350+0000 7fcd1fcc4640 1 --2- 172.21.15.87:0/1806398154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcd18070c40 0x7fcd180ae600 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.350+0000 7fcd1fcc4640 1 -- 172.21.15.87:0/1806398154 >> 172.21.15.87:0/1806398154 conn(0x7fcd1806d440 msgr2=0x7fcd180b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.351+0000 7fcd1fcc4640 1 -- 172.21.15.87:0/1806398154 shutdown_connections 2024-09-17T03:13:49.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.351+0000 7fcd1fcc4640 1 -- 172.21.15.87:0/1806398154 wait complete. 2024-09-17T03:13:49.378 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi087') 2024-09-17T03:13:49.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.419+0000 7f90bcbc6640 1 -- 172.21.15.87:0/1763404853 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90b806ed60 msgr2=0x7f90b80739e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.419+0000 7f90bcbc6640 1 --2- 172.21.15.87:0/1763404853 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90b806ed60 0x7f90b80739e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f90b80b2c80 tx=0x7f90ac02a4f0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.419+0000 7f90bcbc6640 1 -- 172.21.15.87:0/1763404853 shutdown_connections 2024-09-17T03:13:49.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.419+0000 7f90bcbc6640 1 --2- 172.21.15.87:0/1763404853 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90b806ed60 0x7f90b80739e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.419+0000 7f90bcbc6640 1 --2- 172.21.15.87:0/1763404853 >> v2:172.21.15.149:3300/0 conn(0x7f90b8070e90 0x7f90b81517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.419+0000 7f90bcbc6640 1 -- 172.21.15.87:0/1763404853 >> 172.21.15.87:0/1763404853 conn(0x7f90b80b1c40 msgr2=0x7f90b80b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.420+0000 7f90bcbc6640 1 -- 172.21.15.87:0/1763404853 shutdown_connections 2024-09-17T03:13:49.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.420+0000 7f90bcbc6640 1 -- 172.21.15.87:0/1763404853 wait complete. 2024-09-17T03:13:49.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.424+0000 7f90bcbc6640 1 Processor -- start 2024-09-17T03:13:49.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.440+0000 7f90bcbc6640 1 -- start start 2024-09-17T03:13:49.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.442+0000 7f90bcbc6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90b806ed60 0x7f90b814ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.443+0000 7f90b659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90b806ed60 0x7f90b814ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.443+0000 7f90b659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90b806ed60 0x7f90b814ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40858/0 (socket says 172.21.15.87:40858) 2024-09-17T03:13:49.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.443+0000 7f90b659c640 1 -- 172.21.15.87:0/3864808423 learned_addr learned my addr 172.21.15.87:0/3864808423 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:13:49.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.443+0000 7f90bcbc6640 1 --2- 172.21.15.87:0/3864808423 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90b806f170 0x7f90b8154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.443+0000 7f90bcbc6640 1 --2- 172.21.15.87:0/3864808423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90b8070e90 0x7f90b8154590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.443+0000 7f90bcbc6640 1 -- 172.21.15.87:0/3864808423 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f90b80742b0 con 0x7f90b806ed60 2024-09-17T03:13:49.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.443+0000 7f90b5d9b640 1 --2- 172.21.15.87:0/3864808423 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90b806f170 0x7f90b8154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.443+0000 7f90b6d9d640 1 --2- 172.21.15.87:0/3864808423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90b8070e90 0x7f90b8154590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.444+0000 7f90bcbc6640 1 -- 172.21.15.87:0/3864808423 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f90b8074130 con 0x7f90b8070e90 2024-09-17T03:13:49.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.444+0000 7f90bcbc6640 1 -- 172.21.15.87:0/3864808423 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f90b80b3d50 con 0x7f90b806f170 2024-09-17T03:13:49.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.444+0000 7f90b659c640 1 -- 172.21.15.87:0/3864808423 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90b806f170 msgr2=0x7f90b8154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.444+0000 7f90b659c640 1 --2- 172.21.15.87:0/3864808423 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90b806f170 0x7f90b8154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.444+0000 7f90b659c640 1 -- 172.21.15.87:0/3864808423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90b8070e90 msgr2=0x7f90b8154590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.444+0000 7f90b659c640 1 --2- 172.21.15.87:0/3864808423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90b8070e90 0x7f90b8154590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.444+0000 7f90b659c640 1 -- 172.21.15.87:0/3864808423 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b815dda0 con 0x7f90b806ed60 2024-09-17T03:13:49.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.445+0000 7f90b659c640 1 --2- 172.21.15.87:0/3864808423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90b806ed60 0x7f90b814ff90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f90a800b780 tx=0x7f90a800bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:49.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.445+0000 7f90b5d9b640 1 --2- 172.21.15.87:0/3864808423 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90b806f170 0x7f90b8154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:13:49.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.445+0000 7f90b6d9d640 1 --2- 172.21.15.87:0/3864808423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90b8070e90 0x7f90b8154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:13:49.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.446+0000 7f90a6ffd640 1 -- 172.21.15.87:0/3864808423 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8019070 con 0x7f90b806ed60 2024-09-17T03:13:49.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.446+0000 7f90a6ffd640 1 -- 172.21.15.87:0/3864808423 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90a800c810 con 0x7f90b806ed60 2024-09-17T03:13:49.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.446+0000 7f90bcbc6640 1 -- 172.21.15.87:0/3864808423 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b8202320 con 0x7f90b806ed60 2024-09-17T03:13:49.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.446+0000 7f90bcbc6640 1 -- 172.21.15.87:0/3864808423 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b82028e0 con 0x7f90b806ed60 2024-09-17T03:13:49.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.448+0000 7f90a6ffd640 1 -- 172.21.15.87:0/3864808423 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a800cc60 con 0x7f90b806ed60 2024-09-17T03:13:49.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.449+0000 7f90a6ffd640 1 -- 172.21.15.87:0/3864808423 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f90a801c070 con 0x7f90b806ed60 2024-09-17T03:13:49.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.450+0000 7f90a6ffd640 1 -- 172.21.15.87:0/3864808423 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f90a801d980 con 0x7f90b806ed60 2024-09-17T03:13:49.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.450+0000 7f90bcbc6640 1 -- 172.21.15.87:0/3864808423 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f909c005290 con 0x7f90b806ed60 2024-09-17T03:13:49.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.454+0000 7f90a6ffd640 1 -- 172.21.15.87:0/3864808423 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f90a800ce00 con 0x7f90b806ed60 2024-09-17T03:13:49.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.514+0000 7f2dc59ad640 1 -- 172.21.15.87:0/3159833144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2dc0073540 msgr2=0x7f2dc00af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.514+0000 7f2dc59ad640 1 --2- 172.21.15.87:0/3159833144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2dc0073540 0x7f2dc00af190 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0071a60 tx=0x7f2db002a3d0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.514+0000 7f2dc59ad640 1 -- 172.21.15.87:0/3159833144 shutdown_connections 2024-09-17T03:13:49.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.514+0000 7f2dc59ad640 1 --2- 172.21.15.87:0/3159833144 >> v2:172.21.15.87:3300/0 conn(0x7f2dc016b6d0 0x7f2dc016bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.514+0000 7f2dc59ad640 1 --2- 172.21.15.87:0/3159833144 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2dc0073540 0x7f2dc00af190 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.515+0000 7f2dc59ad640 1 -- 172.21.15.87:0/3159833144 >> 172.21.15.87:0/3159833144 conn(0x7f2dc00b1b60 msgr2=0x7f2dc00b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.515+0000 7f2dc59ad640 1 -- 172.21.15.87:0/3159833144 shutdown_connections 2024-09-17T03:13:49.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.515+0000 7f2dc59ad640 1 -- 172.21.15.87:0/3159833144 wait complete. 2024-09-17T03:13:49.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.518+0000 7f2dc59ad640 1 Processor -- start 2024-09-17T03:13:49.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.519+0000 7f2dc59ad640 1 -- start start 2024-09-17T03:13:49.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.521+0000 7f2dc59ad640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2dc00cee70 0x7f2dc0215950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.521+0000 7f2dc59ad640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2dc016b6d0 0x7f2dc0216280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.521+0000 7f2dbe7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2dc016b6d0 0x7f2dc0216280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.521+0000 7f2dbe7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2dc016b6d0 0x7f2dc0216280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40870/0 (socket says 172.21.15.87:40870) 2024-09-17T03:13:49.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.522+0000 7f2dbeffd640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2dc00cee70 0x7f2dc0215950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.522+0000 7f2dbe7fc640 1 -- 172.21.15.87:0/1117947764 learned_addr learned my addr 172.21.15.87:0/1117947764 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:13:49.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.522+0000 7f2dc59ad640 1 --2- 172.21.15.87:0/1117947764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2dc02167c0 0x7f2dc0178290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.522+0000 7f2dc59ad640 1 -- 172.21.15.87:0/1117947764 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc008bd30 con 0x7f2dc016b6d0 2024-09-17T03:13:49.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.522+0000 7f2dc59ad640 1 -- 172.21.15.87:0/1117947764 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc008bbb0 con 0x7f2dc02167c0 2024-09-17T03:13:49.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.522+0000 7f2dc59ad640 1 -- 172.21.15.87:0/1117947764 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f2dc016c020 con 0x7f2dc00cee70 2024-09-17T03:13:49.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.523+0000 7f2dbe7fc640 1 -- 172.21.15.87:0/1117947764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2dc00cee70 msgr2=0x7f2dc0215950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.523+0000 7f2dbe7fc640 1 --2- 172.21.15.87:0/1117947764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2dc00cee70 0x7f2dc0215950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.523+0000 7f2dbe7fc640 1 -- 172.21.15.87:0/1117947764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2dc02167c0 msgr2=0x7f2dc0178290 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T03:13:49.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.523+0000 7f2dbe7fc640 1 --2- 172.21.15.87:0/1117947764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2dc02167c0 0x7f2dc0178290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.523+0000 7f2dbe7fc640 1 -- 172.21.15.87:0/1117947764 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc0216bd0 con 0x7f2dc016b6d0 2024-09-17T03:13:49.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.523+0000 7f2dbe7fc640 1 --2- 172.21.15.87:0/1117947764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2dc016b6d0 0x7f2dc0216280 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2db800cb60 tx=0x7f2db800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:49.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.524+0000 7f2da7fff640 1 -- 172.21.15.87:0/1117947764 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db8017070 con 0x7f2dc016b6d0 2024-09-17T03:13:49.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.524+0000 7f2da7fff640 1 -- 172.21.15.87:0/1117947764 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2db800abd0 con 0x7f2dc016b6d0 2024-09-17T03:13:49.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.525+0000 7f2dc59ad640 1 -- 172.21.15.87:0/1117947764 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dc01771f0 con 0x7f2dc016b6d0 2024-09-17T03:13:49.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.525+0000 7f2dc59ad640 1 -- 172.21.15.87:0/1117947764 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dc01777b0 con 0x7f2dc016b6d0 2024-09-17T03:13:49.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.527+0000 7f2da7fff640 1 -- 172.21.15.87:0/1117947764 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db8003970 con 0x7f2dc016b6d0 2024-09-17T03:13:49.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.527+0000 7f2da7fff640 1 -- 172.21.15.87:0/1117947764 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2db801a070 con 0x7f2dc016b6d0 2024-09-17T03:13:49.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.524+0000 7f22c97fa640 1 -- 172.21.15.87:0/1781433136 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f22c4005580 con 0x7f22e4071ba0 2024-09-17T03:13:49.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.528+0000 7f2da7fff640 1 -- 172.21.15.87:0/1117947764 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2db80128c0 con 0x7f2dc016b6d0 2024-09-17T03:13:49.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.529+0000 7f2da5ffb640 1 -- 172.21.15.87:0/1117947764 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2da0005290 con 0x7f2dc016b6d0 2024-09-17T03:13:49.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.536+0000 7f22cb7fe640 1 -- 172.21.15.87:0/1781433136 <== mon.0 v2:172.21.15.87:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v8)=0 v8) ==== 117+0+0 (secure 0 0 0) 0x7f22dc046600 con 0x7f22e4071ba0 2024-09-17T03:13:49.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.539+0000 7f22c97fa640 1 -- 172.21.15.87:0/1781433136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22e4071ba0 msgr2=0x7f22e4154290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.539+0000 7f22c97fa640 1 --2- 172.21.15.87:0/1781433136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22e4071ba0 0x7f22e4154290 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f22dc009000 tx=0x7f22dc004a40 comp rx=0 tx=0).stop 2024-09-17T03:13:49.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.539+0000 7f2da7fff640 1 -- 172.21.15.87:0/1117947764 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f2dc0216bd0 con 0x7f2dc016b6d0 2024-09-17T03:13:49.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.540+0000 7f22c97fa640 1 -- 172.21.15.87:0/1781433136 shutdown_connections 2024-09-17T03:13:49.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.540+0000 7f22c97fa640 1 --2- 172.21.15.87:0/1781433136 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22e4071ba0 0x7f22e4154290 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.540+0000 7f22c97fa640 1 --2- 172.21.15.87:0/1781433136 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22e415f0a0 0x7f22e40b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.540+0000 7f22c97fa640 1 --2- 172.21.15.87:0/1781433136 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22e415c5e0 0x7f22e4205fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.540+0000 7f22c97fa640 1 -- 172.21.15.87:0/1781433136 >> 172.21.15.87:0/1781433136 conn(0x7f22e40705d0 msgr2=0x7f22e4073db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.544+0000 7f22c97fa640 1 -- 172.21.15.87:0/1781433136 shutdown_connections 2024-09-17T03:13:49.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.544+0000 7f22c97fa640 1 -- 172.21.15.87:0/1781433136 wait complete. 2024-09-17T03:13:49.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.572+0000 7f5be9cde640 1 -- 172.21.15.87:0/2549395474 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5be40732d0 msgr2=0x7f5be40736e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.572+0000 7f5be9cde640 1 --2- 172.21.15.87:0/2549395474 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5be40732d0 0x7f5be40736e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc005fd0 tx=0x7f5bdc03b7f0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.572+0000 7f5be9cde640 1 -- 172.21.15.87:0/2549395474 shutdown_connections 2024-09-17T03:13:49.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.572+0000 7f5be9cde640 1 --2- 172.21.15.87:0/2549395474 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5be40732d0 0x7f5be40736e0 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc005fd0 tx=0x7f5bdc03b7f0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.572+0000 7f5be9cde640 1 -- 172.21.15.87:0/2549395474 >> 172.21.15.87:0/2549395474 conn(0x7f5be40b1b60 msgr2=0x7f5be40b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.576+0000 7f5be9cde640 1 -- 172.21.15.87:0/2549395474 shutdown_connections 2024-09-17T03:13:49.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.576+0000 7f5be9cde640 1 -- 172.21.15.87:0/2549395474 wait complete. 2024-09-17T03:13:49.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.579+0000 7f5be9cde640 1 Processor -- start 2024-09-17T03:13:49.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.579+0000 7f5be9cde640 1 -- start start 2024-09-17T03:13:49.581 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi087') 2024-09-17T03:13:49.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.582+0000 7f5be9cde640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5be406fa80 0x7f5be4148b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.582+0000 7f5be9cde640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5be40ace40 0x7f5be415d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.583+0000 7f5be9cde640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5be415e3d0 0x7f5be415e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.582+0000 7f5be37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5be406fa80 0x7f5be4148b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.583+0000 7f5be37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5be406fa80 0x7f5be4148b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40896/0 (socket says 172.21.15.87:40896) 2024-09-17T03:13:49.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.583+0000 7f5be2ffd640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5be40ace40 0x7f5be415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.583+0000 7f5be2ffd640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5be40ace40 0x7f5be415d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.87:53698/0 (socket says 172.21.15.87:53698) 2024-09-17T03:13:49.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.583+0000 7f5be9cde640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5be415ce90 con 0x7f5be406fa80 2024-09-17T03:13:49.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.583+0000 7f5be9cde640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5be415cd10 con 0x7f5be415e3d0 2024-09-17T03:13:49.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.583+0000 7f5be9cde640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5be415d010 con 0x7f5be40ace40 2024-09-17T03:13:49.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.583+0000 7f5be3fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5be415e3d0 0x7f5be415e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.583+0000 7f5be3fff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5be415e3d0 0x7f5be415e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.87:33574/0 (socket says 172.21.15.87:33574) 2024-09-17T03:13:49.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.583+0000 7f5be3fff640 1 -- 172.21.15.87:0/796738449 learned_addr learned my addr 172.21.15.87:0/796738449 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:13:49.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.584+0000 7f5be3fff640 1 -- 172.21.15.87:0/796738449 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5be40ace40 msgr2=0x7f5be415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.584+0000 7f5be3fff640 1 --2- 172.21.15.87:0/796738449 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5be40ace40 0x7f5be415d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.584+0000 7f5be3fff640 1 -- 172.21.15.87:0/796738449 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5be406fa80 msgr2=0x7f5be4148b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.584+0000 7f5be3fff640 1 --2- 172.21.15.87:0/796738449 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5be406fa80 0x7f5be4148b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.584+0000 7f5be3fff640 1 -- 172.21.15.87:0/796738449 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be415ed20 con 0x7f5be415e3d0 2024-09-17T03:13:49.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.584+0000 7f5be37fe640 1 --2- 172.21.15.87:0/796738449 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5be406fa80 0x7f5be4148b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:13:49.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.585+0000 7f5be3fff640 1 --2- 172.21.15.87:0/796738449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5be415e3d0 0x7f5be415e7e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800c500 tx=0x7f5bd800c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:49.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.585+0000 7f5bcbfff640 1 -- 172.21.15.87:0/796738449 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8017070 con 0x7f5be415e3d0 2024-09-17T03:13:49.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.585+0000 7f5bcbfff640 1 -- 172.21.15.87:0/796738449 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bd80036e0 con 0x7f5be415e3d0 2024-09-17T03:13:49.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.586+0000 7f5bcbfff640 1 -- 172.21.15.87:0/796738449 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8003b30 con 0x7f5be415e3d0 2024-09-17T03:13:49.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.586+0000 7f5be2ffd640 1 --2- 172.21.15.87:0/796738449 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5be40ace40 0x7f5be415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:13:49.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.586+0000 7f5be9cde640 1 -- 172.21.15.87:0/796738449 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be409b030 con 0x7f5be415e3d0 2024-09-17T03:13:49.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.587+0000 7f5be9cde640 1 -- 172.21.15.87:0/796738449 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be409b5f0 con 0x7f5be415e3d0 2024-09-17T03:13:49.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.587+0000 7f5bcbfff640 1 -- 172.21.15.87:0/796738449 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5bd801a070 con 0x7f5be415e3d0 2024-09-17T03:13:49.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.587+0000 7f5bcbfff640 1 -- 172.21.15.87:0/796738449 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5bd8012740 con 0x7f5be415e3d0 2024-09-17T03:13:49.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.587+0000 7f3a2d7e4640 1 -- 172.21.15.87:0/3772474109 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3a10003e80 msgr2=0x7f3a100042d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.587+0000 7f3a2d7e4640 1 --2- 172.21.15.87:0/3772474109 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3a10003e80 0x7f3a100042d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3a18006f70 tx=0x7f3a10003b10 comp rx=0 tx=0).stop 2024-09-17T03:13:49.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.587+0000 7f3a2d7e4640 1 -- 172.21.15.87:0/3772474109 shutdown_connections 2024-09-17T03:13:49.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.587+0000 7f3a2d7e4640 1 --2- 172.21.15.87:0/3772474109 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3a10003e80 0x7f3a100042d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.587+0000 7f3a2d7e4640 1 -- 172.21.15.87:0/3772474109 >> 172.21.15.87:0/3772474109 conn(0x7f3a280b1b60 msgr2=0x7f3a280b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.589+0000 7f5bc9ffb640 1 -- 172.21.15.87:0/796738449 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be40736e0 con 0x7f5be415e3d0 2024-09-17T03:13:49.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.591+0000 7f3a2d7e4640 1 -- 172.21.15.87:0/3772474109 shutdown_connections 2024-09-17T03:13:49.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.591+0000 7f3a2d7e4640 1 -- 172.21.15.87:0/3772474109 wait complete. 2024-09-17T03:13:49.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.594+0000 7f3a2d7e4640 1 Processor -- start 2024-09-17T03:13:49.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.594+0000 7f3a2d7e4640 1 -- start start 2024-09-17T03:13:49.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.597+0000 7f3a2d7e4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a28205e80 0x7f3a2809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.597+0000 7f3a26ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a28205e80 0x7f3a2809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.597+0000 7f3a26ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a28205e80 0x7f3a2809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40908/0 (socket says 172.21.15.87:40908) 2024-09-17T03:13:49.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.597+0000 7f3a26ffd640 1 -- 172.21.15.87:0/1151270359 learned_addr learned my addr 172.21.15.87:0/1151270359 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:13:49.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.598+0000 7f3a2d7e4640 1 --2- 172.21.15.87:0/1151270359 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3a280c4dd0 0x7f3a282056f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.598+0000 7f3a2d7e4640 1 --2- 172.21.15.87:0/1151270359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a28208240 0x7f3a28208650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.598+0000 7f3a2d7e4640 1 -- 172.21.15.87:0/1151270359 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2808bd30 con 0x7f3a28205e80 2024-09-17T03:13:49.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.598+0000 7f3a2d7e4640 1 -- 172.21.15.87:0/1151270359 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2808bbb0 con 0x7f3a28208240 2024-09-17T03:13:49.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.598+0000 7f3a2d7e4640 1 -- 172.21.15.87:0/1151270359 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3a2816bae0 con 0x7f3a280c4dd0 2024-09-17T03:13:49.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.599+0000 7f3a277fe640 1 --2- 172.21.15.87:0/1151270359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a28208240 0x7f3a28208650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.600+0000 7f3a26ffd640 1 -- 172.21.15.87:0/1151270359 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3a280c4dd0 msgr2=0x7f3a282056f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T03:13:49.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.600+0000 7f3a26ffd640 1 --2- 172.21.15.87:0/1151270359 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3a280c4dd0 0x7f3a282056f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.600+0000 7f3a26ffd640 1 -- 172.21.15.87:0/1151270359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a28208240 msgr2=0x7f3a28208650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.600+0000 7f3a26ffd640 1 --2- 172.21.15.87:0/1151270359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a28208240 0x7f3a28208650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.600+0000 7f3a26ffd640 1 -- 172.21.15.87:0/1151270359 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a28208b90 con 0x7f3a28205e80 2024-09-17T03:13:49.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.600+0000 7f3a26ffd640 1 --2- 172.21.15.87:0/1151270359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a28205e80 0x7f3a2809c480 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3a18000c00 tx=0x7f3a180038f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:49.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.601+0000 7f3a0f7fe640 1 -- 172.21.15.87:0/1151270359 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a18003ce0 con 0x7f3a28205e80 2024-09-17T03:13:49.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.601+0000 7f3a0f7fe640 1 -- 172.21.15.87:0/1151270359 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a1802e480 con 0x7f3a28205e80 2024-09-17T03:13:49.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.601+0000 7f3a2d7e4640 1 -- 172.21.15.87:0/1151270359 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a280c5730 con 0x7f3a28205e80 2024-09-17T03:13:49.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.601+0000 7f5bcbfff640 1 -- 172.21.15.87:0/796738449 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f5bd8021020 con 0x7f5be415e3d0 2024-09-17T03:13:49.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.601+0000 7f3a2d7e4640 1 -- 172.21.15.87:0/1151270359 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a282140d0 con 0x7f3a28205e80 2024-09-17T03:13:49.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.601+0000 7f65c2263640 1 -- 172.21.15.87:0/2167581244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65bc073540 msgr2=0x7f65bc0af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.601+0000 7f65c2263640 1 --2- 172.21.15.87:0/2167581244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65bc073540 0x7f65bc0af190 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f65bc071a60 tx=0x7f65b402a740 comp rx=0 tx=0).stop 2024-09-17T03:13:49.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.601+0000 7f65c2263640 1 -- 172.21.15.87:0/2167581244 shutdown_connections 2024-09-17T03:13:49.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.601+0000 7f65c2263640 1 --2- 172.21.15.87:0/2167581244 >> v2:172.21.15.87:3300/0 conn(0x7f65bc16b6d0 0x7f65bc16bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.601+0000 7f65c2263640 1 --2- 172.21.15.87:0/2167581244 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65bc073540 0x7f65bc0af190 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f65bc071a60 tx=0x7f65b402a740 comp rx=0 tx=0).stop 2024-09-17T03:13:49.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.601+0000 7f65c2263640 1 -- 172.21.15.87:0/2167581244 >> 172.21.15.87:0/2167581244 conn(0x7f65bc0b1b60 msgr2=0x7f65bc0b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.602+0000 7f3a0f7fe640 1 -- 172.21.15.87:0/1151270359 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1802e890 con 0x7f3a28205e80 2024-09-17T03:13:49.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.607+0000 7f3a0f7fe640 1 -- 172.21.15.87:0/1151270359 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3a18035020 con 0x7f3a28205e80 2024-09-17T03:13:49.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.608+0000 7f2d0a007640 1 -- 172.21.15.87:0/3684381703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d040cee70 msgr2=0x7f2d040cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.608+0000 7f2d0a007640 1 --2- 172.21.15.87:0/3684381703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d040cee70 0x7f2d040cf280 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4005fd0 tx=0x7f2cf402a510 comp rx=0 tx=0).stop 2024-09-17T03:13:49.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.608+0000 7f2d0a007640 1 -- 172.21.15.87:0/3684381703 shutdown_connections 2024-09-17T03:13:49.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.608+0000 7f2d0a007640 1 --2- 172.21.15.87:0/3684381703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d040cee70 0x7f2d040cf280 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.608+0000 7f2d0a007640 1 --2- 172.21.15.87:0/3684381703 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2d04073540 0x7f2d040af190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.608+0000 7f2d0a007640 1 -- 172.21.15.87:0/3684381703 >> 172.21.15.87:0/3684381703 conn(0x7f2d040b1b60 msgr2=0x7f2d040b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.611+0000 7f65c2263640 1 -- 172.21.15.87:0/2167581244 shutdown_connections 2024-09-17T03:13:49.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.612+0000 7f3a0f7fe640 1 -- 172.21.15.87:0/1151270359 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3a18037cc0 con 0x7f3a28205e80 2024-09-17T03:13:49.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.612+0000 7f3a0d7fa640 1 -- 172.21.15.87:0/1151270359 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a08005290 con 0x7f3a28205e80 2024-09-17T03:13:49.616 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.615+0000 7f14e9ffb640 1 -- 172.21.15.87:0/1059120084 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f14e4005850 con 0x7f150416b6d0 2024-09-17T03:13:49.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.612+0000 7f65c2263640 1 -- 172.21.15.87:0/2167581244 wait complete. 2024-09-17T03:13:49.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.614+0000 7f65c2263640 1 Processor -- start 2024-09-17T03:13:49.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.615+0000 7f65c2263640 1 -- start start 2024-09-17T03:13:49.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.617+0000 7f65c2263640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65bc0cee70 0x7f65bc15fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.617+0000 7f65c2263640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f65bc16b6d0 0x7f65bc177e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.617+0000 7f65c2263640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65bc162710 0x7f65bc178380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.617+0000 7f65c0800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65bc162710 0x7f65bc178380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.617+0000 7f65c0800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65bc162710 0x7f65bc178380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40934/0 (socket says 172.21.15.87:40934) 2024-09-17T03:13:49.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.617+0000 7f65c0800640 1 -- 172.21.15.87:0/1980948656 learned_addr learned my addr 172.21.15.87:0/1980948656 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:13:49.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.618+0000 7f65c2263640 1 -- 172.21.15.87:0/1980948656 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc08bd30 con 0x7f65bc162710 2024-09-17T03:13:49.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.618+0000 7f65c2263640 1 -- 172.21.15.87:0/1980948656 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f65bc08bbb0 con 0x7f65bc0cee70 2024-09-17T03:13:49.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.618+0000 7f65c2263640 1 -- 172.21.15.87:0/1980948656 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f65bc16c020 con 0x7f65bc16b6d0 2024-09-17T03:13:49.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.619+0000 7f2d0a007640 1 -- 172.21.15.87:0/3684381703 shutdown_connections 2024-09-17T03:13:49.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.618+0000 7f65c0800640 1 -- 172.21.15.87:0/1980948656 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f65bc16b6d0 msgr2=0x7f65bc177e40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T03:13:49.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.618+0000 7f65c0800640 1 --2- 172.21.15.87:0/1980948656 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f65bc16b6d0 0x7f65bc177e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.618+0000 7f65c0800640 1 -- 172.21.15.87:0/1980948656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65bc0cee70 msgr2=0x7f65bc15fad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T03:13:49.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.618+0000 7f65c0800640 1 --2- 172.21.15.87:0/1980948656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65bc0cee70 0x7f65bc15fad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.618+0000 7f65c0800640 1 -- 172.21.15.87:0/1980948656 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65bc160010 con 0x7f65bc162710 2024-09-17T03:13:49.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.619+0000 7f65c0800640 1 --2- 172.21.15.87:0/1980948656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65bc162710 0x7f65bc178380 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f65b000c9e0 tx=0x7f65b000cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:49.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.624+0000 7f7902d8a640 1 -- 172.21.15.87:0/2565007635 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f78fc0b47b0 con 0x7f78fc0afc40 2024-09-17T03:13:49.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.621+0000 7f3a0f7fe640 1 -- 172.21.15.87:0/1151270359 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f3a18047060 con 0x7f3a28205e80 2024-09-17T03:13:49.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.620+0000 7f2d0a007640 1 -- 172.21.15.87:0/3684381703 wait complete. 2024-09-17T03:13:49.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.622+0000 7f2d0a007640 1 Processor -- start 2024-09-17T03:13:49.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.621+0000 7f65b8ff9640 1 -- 172.21.15.87:0/1980948656 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b0007a30 con 0x7f65bc162710 2024-09-17T03:13:49.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.621+0000 7f65b8ff9640 1 -- 172.21.15.87:0/1980948656 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65b0007ce0 con 0x7f65bc162710 2024-09-17T03:13:49.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.622+0000 7f65c2263640 1 -- 172.21.15.87:0/1980948656 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65bc162eb0 con 0x7f65bc162710 2024-09-17T03:13:49.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.622+0000 7f65c2263640 1 -- 172.21.15.87:0/1980948656 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65bc163b30 con 0x7f65bc162710 2024-09-17T03:13:49.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.622+0000 7f65b8ff9640 1 -- 172.21.15.87:0/1980948656 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65b000f6b0 con 0x7f65bc162710 2024-09-17T03:13:49.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.623+0000 7f65b8ff9640 1 -- 172.21.15.87:0/1980948656 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f65b000f850 con 0x7f65bc162710 2024-09-17T03:13:49.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.623+0000 7f65b8ff9640 1 -- 172.21.15.87:0/1980948656 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f65b0004b80 con 0x7f65bc162710 2024-09-17T03:13:49.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.624+0000 7f65c2263640 1 -- 172.21.15.87:0/1980948656 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f659c005290 con 0x7f65bc162710 2024-09-17T03:13:49.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.623+0000 7f2d0a007640 1 -- start start 2024-09-17T03:13:49.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.625+0000 7f2d0a007640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d04073540 0x7f2d040c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.625+0000 7f2d0a007640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d0416b6d0 0x7f2d04205040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.625+0000 7f2d0a007640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2d0409c650 0x7f2d0409ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.625+0000 7f2d0a007640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0408bbb0 con 0x7f2d04073540 2024-09-17T03:13:49.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.625+0000 7f2d0a007640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2d040cf8d0 con 0x7f2d0416b6d0 2024-09-17T03:13:49.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.626+0000 7f2d03fff640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2d0409c650 0x7f2d0409ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.626+0000 7f2d03fff640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2d0409c650 0x7f2d0409ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.87:53754/0 (socket says 172.21.15.87:53754) 2024-09-17T03:13:49.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.626+0000 7f2d03fff640 1 -- 172.21.15.87:0/3259846363 learned_addr learned my addr 172.21.15.87:0/3259846363 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:13:49.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.626+0000 7f2d0a007640 1 -- 172.21.15.87:0/3259846363 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f2d0408bd30 con 0x7f2d0409c650 2024-09-17T03:13:49.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.628+0000 7f14ebfff640 1 -- 172.21.15.87:0/1059120084 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v9)=0 v9) ==== 115+0+0 (secure 0 0 0) 0x7f14fc037440 con 0x7f150416b6d0 2024-09-17T03:13:49.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.628+0000 7f2d03fff640 1 -- 172.21.15.87:0/3259846363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d0416b6d0 msgr2=0x7f2d04205040 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T03:13:49.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.628+0000 7f2d03fff640 1 --2- 172.21.15.87:0/3259846363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d0416b6d0 0x7f2d04205040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.628+0000 7f2d03fff640 1 -- 172.21.15.87:0/3259846363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d04073540 msgr2=0x7f2d040c5770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T03:13:49.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.628+0000 7f2d03fff640 1 --2- 172.21.15.87:0/3259846363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d04073540 0x7f2d040c5770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.628+0000 7f2d03fff640 1 -- 172.21.15.87:0/3259846363 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0409cfa0 con 0x7f2d0409c650 2024-09-17T03:13:49.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.632+0000 7f2d03fff640 1 --2- 172.21.15.87:0/3259846363 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2d0409c650 0x7f2d0409ca60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800b410 tx=0x7f2cf800b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:49.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.632+0000 7f2cebfff640 1 -- 172.21.15.87:0/3259846363 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8015020 con 0x7f2d0409c650 2024-09-17T03:13:49.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.632+0000 7f2cebfff640 1 -- 172.21.15.87:0/3259846363 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cf8011920 con 0x7f2d0409c650 2024-09-17T03:13:49.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.632+0000 7f2d0a007640 1 -- 172.21.15.87:0/3259846363 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d040cefa0 con 0x7f2d0409c650 2024-09-17T03:13:49.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.632+0000 7f2d0a007640 1 -- 172.21.15.87:0/3259846363 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d040c5300 con 0x7f2d0409c650 2024-09-17T03:13:49.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.634+0000 7f2cebfff640 1 -- 172.21.15.87:0/3259846363 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8011d70 con 0x7f2d0409c650 2024-09-17T03:13:49.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.635+0000 7f2cebfff640 1 -- 172.21.15.87:0/3259846363 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2cf8018020 con 0x7f2d0409c650 2024-09-17T03:13:49.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.635+0000 7f2cebfff640 1 -- 172.21.15.87:0/3259846363 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2cf801a6f0 con 0x7f2d0409c650 2024-09-17T03:13:49.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.636+0000 7f2ce9ffb640 1 -- 172.21.15.87:0/3259846363 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ce4005290 con 0x7f2d0409c650 2024-09-17T03:13:49.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.638+0000 7f1509cde640 1 -- 172.21.15.87:0/1059120084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f150416b6d0 msgr2=0x7f15041f89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.638+0000 7f1509cde640 1 --2- 172.21.15.87:0/1059120084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f150416b6d0 0x7f15041f89e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f14fc036000 tx=0x7f14fc030b40 comp rx=0 tx=0).stop 2024-09-17T03:13:49.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.638+0000 7f1509cde640 1 -- 172.21.15.87:0/1059120084 shutdown_connections 2024-09-17T03:13:49.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.638+0000 7f1509cde640 1 --2- 172.21.15.87:0/1059120084 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f15041f7510 0x7f15041f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.638+0000 7f1509cde640 1 --2- 172.21.15.87:0/1059120084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f150416b6d0 0x7f15041f89e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.638+0000 7f1509cde640 1 --2- 172.21.15.87:0/1059120084 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1504073540 0x7f15041f3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.638+0000 7f1509cde640 1 -- 172.21.15.87:0/1059120084 >> 172.21.15.87:0/1059120084 conn(0x7f15040b1b60 msgr2=0x7f150416c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.638+0000 7f2cebfff640 1 -- 172.21.15.87:0/3259846363 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f2cf801f070 con 0x7f2d0409c650 2024-09-17T03:13:49.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.647+0000 7fb1181c3640 1 -- 172.21.15.87:0/927628335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb110073540 msgr2=0x7fb1100af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.647+0000 7fb1181c3640 1 --2- 172.21.15.87:0/927628335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb110073540 0x7fb1100af190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb110073ca0 tx=0x7fb10c02a5c0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.647+0000 7fb1181c3640 1 -- 172.21.15.87:0/927628335 shutdown_connections 2024-09-17T03:13:49.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.647+0000 7fb1181c3640 1 --2- 172.21.15.87:0/927628335 >> v2:172.21.15.149:3300/0 conn(0x7fb11016b6d0 0x7fb11016bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.647+0000 7fb1181c3640 1 --2- 172.21.15.87:0/927628335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb110073540 0x7fb1100af190 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.647+0000 7fb1181c3640 1 -- 172.21.15.87:0/927628335 >> 172.21.15.87:0/927628335 conn(0x7fb1100b1b60 msgr2=0x7fb1100b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.644+0000 7f78f97fa640 1 -- 172.21.15.87:0/2565007635 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v10)=0 v10) ==== 106+0+0 (secure 0 0 0) 0x7f78fc0b47b0 con 0x7f78fc0afc40 2024-09-17T03:13:49.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.644+0000 7f1509cde640 1 -- 172.21.15.87:0/1059120084 shutdown_connections 2024-09-17T03:13:49.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.644+0000 7f1509cde640 1 -- 172.21.15.87:0/1059120084 wait complete. 2024-09-17T03:13:49.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.644+0000 7f65b8ff9640 1 -- 172.21.15.87:0/1980948656 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f65b0014030 con 0x7f65bc162710 2024-09-17T03:13:49.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.651+0000 7f7902d8a640 1 -- 172.21.15.87:0/2565007635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78fc0afc40 msgr2=0x7f78fc1f25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.651+0000 7f7902d8a640 1 --2- 172.21.15.87:0/2565007635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78fc0afc40 0x7f78fc1f25d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f78ec00c750 tx=0x7f78ec00cc10 comp rx=0 tx=0).stop 2024-09-17T03:13:49.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.651+0000 7f7902d8a640 1 -- 172.21.15.87:0/2565007635 shutdown_connections 2024-09-17T03:13:49.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.651+0000 7f7902d8a640 1 --2- 172.21.15.87:0/2565007635 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f78fc15f4b0 0x7f78fc0cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.651+0000 7f7902d8a640 1 --2- 172.21.15.87:0/2565007635 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f78fc158780 0x7f78fc1534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.651+0000 7f7902d8a640 1 --2- 172.21.15.87:0/2565007635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78fc0afc40 0x7f78fc1f25d0 secure :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f78ec00c750 tx=0x7f78ec00cc10 comp rx=0 tx=0).stop 2024-09-17T03:13:49.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.651+0000 7f7902d8a640 1 -- 172.21.15.87:0/2565007635 >> 172.21.15.87:0/2565007635 conn(0x7f78fc06d440 msgr2=0x7f78fc159b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.652+0000 7f7902d8a640 1 -- 172.21.15.87:0/2565007635 shutdown_connections 2024-09-17T03:13:49.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.653+0000 7f7902d8a640 1 -- 172.21.15.87:0/2565007635 wait complete. 2024-09-17T03:13:49.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.648+0000 7fb1181c3640 1 -- 172.21.15.87:0/927628335 shutdown_connections 2024-09-17T03:13:49.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.648+0000 7fb1181c3640 1 -- 172.21.15.87:0/927628335 wait complete. 2024-09-17T03:13:49.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.650+0000 7fb1181c3640 1 Processor -- start 2024-09-17T03:13:49.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.653+0000 7fb1181c3640 1 -- start start 2024-09-17T03:13:49.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.656+0000 7fb1181c3640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb1100cee70 0x7fb1100c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.656+0000 7fb1181c3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb11016b6d0 0x7fb11009d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.656+0000 7fb1181c3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb11009d670 0x7fb1100b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.656+0000 7fb1181c3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb11016c1a0 con 0x7fb11009d670 2024-09-17T03:13:49.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.656+0000 7fb1181c3640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb11016c020 con 0x7fb11016b6d0 2024-09-17T03:13:49.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.656+0000 7fb1181c3640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb1100b3180 con 0x7fb1100cee70 2024-09-17T03:13:49.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.657+0000 7fb115f5f640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb1100cee70 0x7fb1100c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.657+0000 7fb11575e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb11016b6d0 0x7fb11009d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.657+0000 7fb11575e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb11016b6d0 0x7fb11009d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.87:33640/0 (socket says 172.21.15.87:33640) 2024-09-17T03:13:49.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.657+0000 7fb11575e640 1 -- 172.21.15.87:0/2982562726 learned_addr learned my addr 172.21.15.87:0/2982562726 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:13:49.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.657+0000 7fb116760640 1 --2- 172.21.15.87:0/2982562726 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb11009d670 0x7fb1100b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.657+0000 7fb11575e640 1 -- 172.21.15.87:0/2982562726 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb1100cee70 msgr2=0x7fb1100c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.657+0000 7fb11575e640 1 --2- 172.21.15.87:0/2982562726 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb1100cee70 0x7fb1100c77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.657+0000 7fb11575e640 1 -- 172.21.15.87:0/2982562726 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb11009d670 msgr2=0x7fb1100b04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.658+0000 7fb11575e640 1 --2- 172.21.15.87:0/2982562726 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb11009d670 0x7fb1100b04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.658+0000 7fb11575e640 1 -- 172.21.15.87:0/2982562726 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb110201fe0 con 0x7fb11016b6d0 2024-09-17T03:13:49.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.658+0000 7fb116760640 1 --2- 172.21.15.87:0/2982562726 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb11009d670 0x7fb1100b04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:13:49.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.658+0000 7fb11575e640 1 --2- 172.21.15.87:0/2982562726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb11016b6d0 0x7fb11009d130 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb10800c7b0 tx=0x7fb10800cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:49.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.659+0000 7fb1067fc640 1 -- 172.21.15.87:0/2982562726 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb108003b70 con 0x7fb11016b6d0 2024-09-17T03:13:49.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.659+0000 7fb1181c3640 1 -- 172.21.15.87:0/2982562726 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1102022c0 con 0x7fb11016b6d0 2024-09-17T03:13:49.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.659+0000 7fb1181c3640 1 -- 172.21.15.87:0/2982562726 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb110202880 con 0x7fb11016b6d0 2024-09-17T03:13:49.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.666+0000 7fb1067fc640 1 -- 172.21.15.87:0/2982562726 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb108003d10 con 0x7fb11016b6d0 2024-09-17T03:13:49.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.666+0000 7fb1067fc640 1 -- 172.21.15.87:0/2982562726 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10800aab0 con 0x7fb11016b6d0 2024-09-17T03:13:49.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.666+0000 7fb1067fc640 1 -- 172.21.15.87:0/2982562726 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb10800ad10 con 0x7fb11016b6d0 2024-09-17T03:13:49.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.666+0000 7fb1067fc640 1 -- 172.21.15.87:0/2982562726 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb108012a30 con 0x7fb11016b6d0 2024-09-17T03:13:49.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.667+0000 7fb1181c3640 1 -- 172.21.15.87:0/2982562726 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0f8005290 con 0x7fb11016b6d0 2024-09-17T03:13:49.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.670+0000 7f05bb920640 1 -- 172.21.15.87:0/3106819927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05b415e220 msgr2=0x7f05b415e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.670+0000 7f05bb920640 1 --2- 172.21.15.87:0/3106819927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05b415e220 0x7f05b415e630 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f05b40b6120 tx=0x7f05ac02a890 comp rx=0 tx=0).stop 2024-09-17T03:13:49.670 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi087') 2024-09-17T03:13:49.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.678+0000 7f05bb920640 1 -- 172.21.15.87:0/3106819927 shutdown_connections 2024-09-17T03:13:49.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.678+0000 7f05bb920640 1 --2- 172.21.15.87:0/3106819927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05b415e220 0x7f05b415e630 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.678+0000 7f05bb920640 1 --2- 172.21.15.87:0/3106819927 >> v2:172.21.15.149:3300/0 conn(0x7f05b40ca580 0x7f05b40ce510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.678+0000 7f05bb920640 1 -- 172.21.15.87:0/3106819927 >> 172.21.15.87:0/3106819927 conn(0x7f05b40cff30 msgr2=0x7f05b40d0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.678 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi087') 2024-09-17T03:13:49.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.678+0000 7fb1067fc640 1 -- 172.21.15.87:0/2982562726 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fb1080050d0 con 0x7fb11016b6d0 2024-09-17T03:13:49.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.684+0000 7f08b0bed640 1 -- 172.21.15.87:0/52337458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0894003a30 msgr2=0x7f0894003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.684+0000 7f08b0bed640 1 --2- 172.21.15.87:0/52337458 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0894003a30 0x7f0894003ea0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f08a0006f80 tx=0x7f0894005830 comp rx=0 tx=0).stop 2024-09-17T03:13:49.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.684+0000 7f08aad9d640 1 -- 172.21.15.87:0/52337458 reap_dead start 2024-09-17T03:13:49.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.684+0000 7f05bb920640 1 -- 172.21.15.87:0/3106819927 shutdown_connections 2024-09-17T03:13:49.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.684+0000 7f05bb920640 1 -- 172.21.15.87:0/3106819927 wait complete. 2024-09-17T03:13:49.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.688+0000 7f05bb920640 1 Processor -- start 2024-09-17T03:13:49.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.688+0000 7f08b0bed640 1 -- 172.21.15.87:0/52337458 shutdown_connections 2024-09-17T03:13:49.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.688+0000 7f08b0bed640 1 -- 172.21.15.87:0/52337458 >> 172.21.15.87:0/52337458 conn(0x7f08a4051450 msgr2=0x7f08a4051890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.688+0000 7f08b0bed640 1 -- 172.21.15.87:0/52337458 shutdown_connections 2024-09-17T03:13:49.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.688+0000 7f08b0bed640 1 -- 172.21.15.87:0/52337458 wait complete. 2024-09-17T03:13:49.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.691+0000 7f08b0bed640 1 Processor -- start 2024-09-17T03:13:49.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.691+0000 7f08b0bed640 1 -- start start 2024-09-17T03:13:49.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.693+0000 7efe38e30640 1 -- 172.21.15.87:0/568520185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe3415b2d0 msgr2=0x7efe340732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.693+0000 7efe38e30640 1 --2- 172.21.15.87:0/568520185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe3415b2d0 0x7efe340732d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7efe24009830 tx=0x7efe2402a360 comp rx=0 tx=0).stop 2024-09-17T03:13:49.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.693+0000 7efe38e30640 1 -- 172.21.15.87:0/568520185 shutdown_connections 2024-09-17T03:13:49.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.693+0000 7efe38e30640 1 --2- 172.21.15.87:0/568520185 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe3415b2d0 0x7efe340732d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.693+0000 7f152d389640 1 -- 172.21.15.87:0/1062310842 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15280afb80 msgr2=0x7f15280ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.693+0000 7f152d389640 1 --2- 172.21.15.87:0/1062310842 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15280afb80 0x7f15280ce7b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f15280d1310 tx=0x7f151c02a440 comp rx=0 tx=0).stop 2024-09-17T03:13:49.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.693+0000 7efe38e30640 1 -- 172.21.15.87:0/568520185 >> 172.21.15.87:0/568520185 conn(0x7efe340d0750 msgr2=0x7efe340d0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.693+0000 7f08b0bed640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0894003a30 0x7f08a40d7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.694+0000 7f152d389640 1 -- 172.21.15.87:0/1062310842 shutdown_connections 2024-09-17T03:13:49.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.694+0000 7f152d389640 1 --2- 172.21.15.87:0/1062310842 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15280afb80 0x7f15280ce7b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.694+0000 7f152d389640 1 -- 172.21.15.87:0/1062310842 >> 172.21.15.87:0/1062310842 conn(0x7f15280b3310 msgr2=0x7f15280b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.694+0000 7efe38e30640 1 -- 172.21.15.87:0/568520185 shutdown_connections 2024-09-17T03:13:49.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.694+0000 7f152d389640 1 -- 172.21.15.87:0/1062310842 shutdown_connections 2024-09-17T03:13:49.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.693+0000 7f05bb920640 1 -- start start 2024-09-17T03:13:49.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.695+0000 7f05bb920640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05b40ca580 0x7f05b41575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.695+0000 7f05bb920640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05b40cc350 0x7f05b4158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.695+0000 7f05bb920640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f05b415e220 0x7f05b4152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.696+0000 7f05bb920640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f05b406f100 con 0x7f05b40ca580 2024-09-17T03:13:49.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.696+0000 7f05bb920640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f05b406ef80 con 0x7f05b40cc350 2024-09-17T03:13:49.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.696+0000 7f05bb920640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f05b406f280 con 0x7f05b415e220 2024-09-17T03:13:49.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.696+0000 7f05b9ebd640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f05b415e220 0x7f05b4152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.694+0000 7efe38e30640 1 -- 172.21.15.87:0/568520185 wait complete. 2024-09-17T03:13:49.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.697+0000 7efe38e30640 1 Processor -- start 2024-09-17T03:13:49.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.694+0000 7f152d389640 1 -- 172.21.15.87:0/1062310842 wait complete. 2024-09-17T03:13:49.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.693+0000 7f08b0bed640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08a40d8030 0x7f08a4034830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.693+0000 7f08b0bed640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08a40d8580 0x7f08a4034d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.693+0000 7f08b0bed640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f08a40ec570 con 0x7f08a40d8580 2024-09-17T03:13:49.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.693+0000 7f08b0bed640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f08a40ec3f0 con 0x7f08a40d8030 2024-09-17T03:13:49.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.693+0000 7f08b0bed640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f08a40ec6f0 con 0x7f0894003a30 2024-09-17T03:13:49.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.696+0000 7f08aad9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08a40d8580 0x7f08a4034d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.696+0000 7f08aad9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08a40d8580 0x7f08a4034d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41004/0 (socket says 172.21.15.87:41004) 2024-09-17T03:13:49.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.696+0000 7f08aad9d640 1 -- 172.21.15.87:0/2274944749 learned_addr learned my addr 172.21.15.87:0/2274944749 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:13:49.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.696+0000 7f08aad9d640 1 -- 172.21.15.87:0/2274944749 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0894003a30 msgr2=0x7f08a40d7830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T03:13:49.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.696+0000 7f08aad9d640 1 --2- 172.21.15.87:0/2274944749 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0894003a30 0x7f08a40d7830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.696+0000 7f08aad9d640 1 -- 172.21.15.87:0/2274944749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08a40d8030 msgr2=0x7f08a4034830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T03:13:49.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.696+0000 7f08aad9d640 1 --2- 172.21.15.87:0/2274944749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08a40d8030 0x7f08a4034830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.696+0000 7f08aad9d640 1 -- 172.21.15.87:0/2274944749 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a40f12e0 con 0x7f08a40d8580 2024-09-17T03:13:49.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.697+0000 7f08aad9d640 1 --2- 172.21.15.87:0/2274944749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08a40d8580 0x7f08a4034d70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f08a40e2be0 tx=0x7f089c009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:49.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.697+0000 7f152d389640 1 Processor -- start 2024-09-17T03:13:49.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.697+0000 7f089affd640 1 -- 172.21.15.87:0/2274944749 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089c016070 con 0x7f08a40d8580 2024-09-17T03:13:49.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.697+0000 7efe38e30640 1 -- start start 2024-09-17T03:13:49.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.698+0000 7f089affd640 1 -- 172.21.15.87:0/2274944749 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f089c0055d0 con 0x7f08a40d8580 2024-09-17T03:13:49.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.698+0000 7f089affd640 1 -- 172.21.15.87:0/2274944749 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089c016070 con 0x7f08a40d8580 2024-09-17T03:13:49.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.699+0000 7fd3829ed640 1 -- 172.21.15.87:0/442379270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd37c08b0b0 msgr2=0x7fd37c08b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.699+0000 7fd3829ed640 1 --2- 172.21.15.87:0/442379270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd37c08b0b0 0x7fd37c08b4c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fd37c06e1b0 tx=0x7fd37402a980 comp rx=0 tx=0).stop 2024-09-17T03:13:49.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.700+0000 7f08b0bed640 1 -- 172.21.15.87:0/2274944749 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a40f15c0 con 0x7f08a40d8580 2024-09-17T03:13:49.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.700+0000 7f089affd640 1 -- 172.21.15.87:0/2274944749 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f089c019070 con 0x7f08a40d8580 2024-09-17T03:13:49.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.696+0000 7f05b9ebd640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f05b415e220 0x7f05b4152740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.87:53794/0 (socket says 172.21.15.87:53794) 2024-09-17T03:13:49.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.696+0000 7f05b9ebd640 1 -- 172.21.15.87:0/3502816569 learned_addr learned my addr 172.21.15.87:0/3502816569 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:13:49.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.696+0000 7f05b8ebb640 1 --2- 172.21.15.87:0/3502816569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05b40cc350 0x7f05b4158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.701+0000 7f08b0bed640 1 -- 172.21.15.87:0/2274944749 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a40f0fb0 con 0x7f08a40d8580 2024-09-17T03:13:49.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.698+0000 7f05b96bc640 1 --2- 172.21.15.87:0/3502816569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05b40ca580 0x7f05b41575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.701+0000 7f05b8ebb640 1 -- 172.21.15.87:0/3502816569 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f05b415e220 msgr2=0x7f05b4152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.701+0000 7f05b8ebb640 1 --2- 172.21.15.87:0/3502816569 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f05b415e220 0x7f05b4152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.701+0000 7f05b8ebb640 1 -- 172.21.15.87:0/3502816569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05b40ca580 msgr2=0x7f05b41575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.701+0000 7f05b8ebb640 1 --2- 172.21.15.87:0/3502816569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05b40ca580 0x7f05b41575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.701+0000 7f05b8ebb640 1 -- 172.21.15.87:0/3502816569 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b40b5ab0 con 0x7f05b40cc350 2024-09-17T03:13:49.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.699+0000 7efe38e30640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe340cca40 0x7efe34201bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.699+0000 7efe38e30640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe3415b2d0 0x7efe34159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.700+0000 7efe31d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe3415b2d0 0x7efe34159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.700+0000 7efe31d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe3415b2d0 0x7efe34159860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41028/0 (socket says 172.21.15.87:41028) 2024-09-17T03:13:49.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.700+0000 7efe31d9b640 1 -- 172.21.15.87:0/4258095832 learned_addr learned my addr 172.21.15.87:0/4258095832 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:13:49.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.700+0000 7efe38e30640 1 --2- 172.21.15.87:0/4258095832 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe3415b6e0 0x7efe34159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.700+0000 7efe38e30640 1 -- 172.21.15.87:0/4258095832 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efe34073810 con 0x7efe3415b2d0 2024-09-17T03:13:49.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.700+0000 7efe38e30640 1 -- 172.21.15.87:0/4258095832 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efe340cf8c0 con 0x7efe340cca40 2024-09-17T03:13:49.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.700+0000 7efe38e30640 1 -- 172.21.15.87:0/4258095832 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7efe34073990 con 0x7efe3415b6e0 2024-09-17T03:13:49.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.700+0000 7efe32d9d640 1 --2- 172.21.15.87:0/4258095832 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe3415b6e0 0x7efe34159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.700+0000 7efe31d9b640 1 -- 172.21.15.87:0/4258095832 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe3415b6e0 msgr2=0x7efe34159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.700+0000 7efe31d9b640 1 --2- 172.21.15.87:0/4258095832 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe3415b6e0 0x7efe34159da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.700+0000 7efe31d9b640 1 -- 172.21.15.87:0/4258095832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe340cca40 msgr2=0x7efe34201bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T03:13:49.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.700+0000 7efe31d9b640 1 --2- 172.21.15.87:0/4258095832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe340cca40 0x7efe34201bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.700+0000 7efe31d9b640 1 -- 172.21.15.87:0/4258095832 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe34159360 con 0x7efe3415b2d0 2024-09-17T03:13:49.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.700+0000 7efe31d9b640 1 --2- 172.21.15.87:0/4258095832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe3415b2d0 0x7efe34159860 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efe24000c00 tx=0x7efe24033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:49.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.701+0000 7efe22ffd640 1 -- 172.21.15.87:0/4258095832 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24040020 con 0x7efe3415b2d0 2024-09-17T03:13:49.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.701+0000 7efe22ffd640 1 -- 172.21.15.87:0/4258095832 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe240035f0 con 0x7efe3415b2d0 2024-09-17T03:13:49.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.701+0000 7efe38e30640 1 -- 172.21.15.87:0/4258095832 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe340b63d0 con 0x7efe3415b2d0 2024-09-17T03:13:49.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.702+0000 7efe38e30640 1 -- 172.21.15.87:0/4258095832 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe34157800 con 0x7efe3415b2d0 2024-09-17T03:13:49.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.702+0000 7fd3829ed640 1 -- 172.21.15.87:0/442379270 shutdown_connections 2024-09-17T03:13:49.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.702+0000 7fd3829ed640 1 --2- 172.21.15.87:0/442379270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd37c08b0b0 0x7fd37c08b4c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.702+0000 7fd3829ed640 1 --2- 172.21.15.87:0/442379270 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd37c0b3b70 0x7fd37c0b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.702+0000 7fd3829ed640 1 --2- 172.21.15.87:0/442379270 >> v2:172.21.15.149:3300/0 conn(0x7fd37c16b430 0x7fd37c0738e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.702+0000 7fd3829ed640 1 -- 172.21.15.87:0/442379270 >> 172.21.15.87:0/442379270 conn(0x7fd37c0b1c40 msgr2=0x7fd37c0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.702+0000 7fd3829ed640 1 -- 172.21.15.87:0/442379270 shutdown_connections 2024-09-17T03:13:49.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.702+0000 7f05b8ebb640 1 --2- 172.21.15.87:0/3502816569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05b40cc350 0x7f05b4158560 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f05b000b440 tx=0x7f05b000b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:49.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.702+0000 7f089affd640 1 -- 172.21.15.87:0/2274944749 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f089c0052e0 con 0x7f08a40d8580 2024-09-17T03:13:49.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.702+0000 7f0898ff9640 1 -- 172.21.15.87:0/2274944749 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0884005290 con 0x7f08a40d8580 2024-09-17T03:13:49.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.698+0000 7f152d389640 1 -- start start 2024-09-17T03:13:49.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.703+0000 7f152d389640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f15280b1310 0x7f1528161cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.703+0000 7f152d389640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15280b3e30 0x7f15281621f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.704+0000 7f1526ffd640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f15280b1310 0x7f1528161cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.704+0000 7f1526ffd640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f15280b1310 0x7f1528161cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.87:53836/0 (socket says 172.21.15.87:53836) 2024-09-17T03:13:49.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.704+0000 7f1526ffd640 1 -- 172.21.15.87:0/1795220589 learned_addr learned my addr 172.21.15.87:0/1795220589 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:13:49.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.704+0000 7f152d389640 1 --2- 172.21.15.87:0/1795220589 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f152815ef00 0x7f1528203f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.704+0000 7f152d389640 1 -- 172.21.15.87:0/1795220589 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f152816e160 con 0x7f15280b3e30 2024-09-17T03:13:49.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.704+0000 7f152d389640 1 -- 172.21.15.87:0/1795220589 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f152816dfe0 con 0x7f152815ef00 2024-09-17T03:13:49.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.704+0000 7f152d389640 1 -- 172.21.15.87:0/1795220589 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f152816e2e0 con 0x7f15280b1310 2024-09-17T03:13:49.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.704+0000 7f15267fc640 1 --2- 172.21.15.87:0/1795220589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15280b3e30 0x7f15281621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.707 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.704+0000 7f15267fc640 1 -- 172.21.15.87:0/1795220589 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f15280b1310 msgr2=0x7f1528161cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.704+0000 7f15267fc640 1 --2- 172.21.15.87:0/1795220589 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f15280b1310 0x7f1528161cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.704+0000 7f15267fc640 1 -- 172.21.15.87:0/1795220589 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f152815ef00 msgr2=0x7f1528203f00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T03:13:49.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.704+0000 7f15267fc640 1 --2- 172.21.15.87:0/1795220589 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f152815ef00 0x7f1528203f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.704+0000 7f15267fc640 1 -- 172.21.15.87:0/1795220589 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f152815d490 con 0x7f15280b3e30 2024-09-17T03:13:49.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.704+0000 7f15267fc640 1 --2- 172.21.15.87:0/1795220589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15280b3e30 0x7f15281621f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f152000c6c0 tx=0x7f152000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:49.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.704+0000 7f05a9ffb640 1 -- 172.21.15.87:0/3502816569 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0015020 con 0x7f05b40cc350 2024-09-17T03:13:49.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.704+0000 7f05bb920640 1 -- 172.21.15.87:0/3502816569 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b4156af0 con 0x7f05b40cc350 2024-09-17T03:13:49.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.705+0000 7f05bb920640 1 -- 172.21.15.87:0/3502816569 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b40b4820 con 0x7f05b40cc350 2024-09-17T03:13:49.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.706+0000 7f05a9ffb640 1 -- 172.21.15.87:0/3502816569 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05b0011940 con 0x7f05b40cc350 2024-09-17T03:13:49.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.706+0000 7f05a9ffb640 1 -- 172.21.15.87:0/3502816569 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0011d90 con 0x7f05b40cc350 2024-09-17T03:13:49.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.704+0000 7efe22ffd640 1 -- 172.21.15.87:0/4258095832 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24037040 con 0x7efe3415b2d0 2024-09-17T03:13:49.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.705+0000 7efe20ff9640 1 -- 172.21.15.87:0/4258095832 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe18005290 con 0x7efe3415b2d0 2024-09-17T03:13:49.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.705+0000 7efe22ffd640 1 -- 172.21.15.87:0/4258095832 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efe24043020 con 0x7efe3415b2d0 2024-09-17T03:13:49.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.703+0000 7fd3829ed640 1 -- 172.21.15.87:0/442379270 wait complete. 2024-09-17T03:13:49.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.705+0000 7fd3829ed640 1 Processor -- start 2024-09-17T03:13:49.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.706+0000 7fd3829ed640 1 -- start start 2024-09-17T03:13:49.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.706+0000 7f05bb920640 1 -- 172.21.15.87:0/3502816569 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05b41626e0 con 0x7f05b40cc350 2024-09-17T03:13:49.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.708+0000 7fd3829ed640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd37c08b0b0 0x7fd37c17caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.708+0000 7f089affd640 1 -- 172.21.15.87:0/2274944749 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f089c021070 con 0x7f08a40d8580 2024-09-17T03:13:49.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.712+0000 7f150f7fe640 1 -- 172.21.15.87:0/1795220589 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1520003b70 con 0x7f15280b3e30 2024-09-17T03:13:49.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.710+0000 7f05a9ffb640 1 -- 172.21.15.87:0/3502816569 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f05b0018020 con 0x7f05b40cc350 2024-09-17T03:13:49.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.710+0000 7f05a9ffb640 1 -- 172.21.15.87:0/3502816569 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f05b0019900 con 0x7f05b40cc350 2024-09-17T03:13:49.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.708+0000 7f90bcbc6640 1 -- 172.21.15.87:0/3864808423 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f909c005580 con 0x7f90b806ed60 2024-09-17T03:13:49.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.709+0000 7efe22ffd640 1 -- 172.21.15.87:0/4258095832 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efe2403cb20 con 0x7efe3415b2d0 2024-09-17T03:13:49.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.708+0000 7fd3829ed640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd37c0b3b70 0x7fd37c15bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.708+0000 7fd3829ed640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd37c16b430 0x7fd37c15c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.708+0000 7fd3829ed640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd37c0ad8f0 con 0x7fd37c0b3b70 2024-09-17T03:13:49.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.708+0000 7fd3829ed640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd37c08be00 con 0x7fd37c16b430 2024-09-17T03:13:49.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.708+0000 7fd3829ed640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd37c0ada70 con 0x7fd37c08b0b0 2024-09-17T03:13:49.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.708+0000 7fd380f8a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd37c16b430 0x7fd37c15c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.708+0000 7fd380f8a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd37c16b430 0x7fd37c15c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.87:33682/0 (socket says 172.21.15.87:33682) 2024-09-17T03:13:49.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.708+0000 7fd380f8a640 1 -- 172.21.15.87:0/2234039483 learned_addr learned my addr 172.21.15.87:0/2234039483 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:13:49.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.709+0000 7fd380f8a640 1 -- 172.21.15.87:0/2234039483 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd37c08b0b0 msgr2=0x7fd37c17caa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T03:13:49.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.709+0000 7fd380f8a640 1 --2- 172.21.15.87:0/2234039483 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd37c08b0b0 0x7fd37c17caa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.709+0000 7fd380f8a640 1 -- 172.21.15.87:0/2234039483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd37c0b3b70 msgr2=0x7fd37c15bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.709+0000 7fd380f8a640 1 --2- 172.21.15.87:0/2234039483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd37c0b3b70 0x7fd37c15bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.709+0000 7fd380f8a640 1 -- 172.21.15.87:0/2234039483 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd37c17cfe0 con 0x7fd37c16b430 2024-09-17T03:13:49.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.712+0000 7f152d389640 1 -- 172.21.15.87:0/1795220589 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f152815d770 con 0x7f15280b3e30 2024-09-17T03:13:49.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.712+0000 7f152d389640 1 -- 172.21.15.87:0/1795220589 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f152815dd30 con 0x7f15280b3e30 2024-09-17T03:13:49.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.713+0000 7fd380f8a640 1 --2- 172.21.15.87:0/2234039483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd37c16b430 0x7fd37c15c120 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd374009d60 tx=0x7fd374003f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:49.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.714+0000 7f05a9ffb640 1 -- 172.21.15.87:0/3502816569 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f05b001f030 con 0x7f05b40cc350 2024-09-17T03:13:49.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.717+0000 7fd378ff9640 1 -- 172.21.15.87:0/2234039483 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd374034040 con 0x7fd37c16b430 2024-09-17T03:13:49.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.717+0000 7fd378ff9640 1 -- 172.21.15.87:0/2234039483 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd374004540 con 0x7fd37c16b430 2024-09-17T03:13:49.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.717+0000 7fd378ff9640 1 -- 172.21.15.87:0/2234039483 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd374045480 con 0x7fd37c16b430 2024-09-17T03:13:49.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.717+0000 7fd3829ed640 1 -- 172.21.15.87:0/2234039483 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd37c0cca40 con 0x7fd37c16b430 2024-09-17T03:13:49.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.717+0000 7fd3829ed640 1 -- 172.21.15.87:0/2234039483 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd37c0ccf40 con 0x7fd37c16b430 2024-09-17T03:13:49.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.717+0000 7fd378ff9640 1 -- 172.21.15.87:0/2234039483 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd374037020 con 0x7fd37c16b430 2024-09-17T03:13:49.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.717+0000 7fd378ff9640 1 -- 172.21.15.87:0/2234039483 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd37403aab0 con 0x7fd37c16b430 2024-09-17T03:13:49.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.722+0000 7f150f7fe640 1 -- 172.21.15.87:0/1795220589 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1520003d10 con 0x7f15280b3e30 2024-09-17T03:13:49.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.722+0000 7f150f7fe640 1 -- 172.21.15.87:0/1795220589 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15200052d0 con 0x7f15280b3e30 2024-09-17T03:13:49.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.722+0000 7efe22ffd640 1 -- 172.21.15.87:0/4258095832 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7efe2404a030 con 0x7efe3415b2d0 2024-09-17T03:13:49.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.723+0000 7fd08fdff640 1 -- 172.21.15.87:0/2852748948 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd074005350 msgr2=0x7fd07400b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.723+0000 7fd08fdff640 1 --2- 172.21.15.87:0/2852748948 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd074005350 0x7fd07400b9e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd08400c390 tx=0x7fd074005830 comp rx=0 tx=0).stop 2024-09-17T03:13:49.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.726+0000 7f150f7fe640 1 -- 172.21.15.87:0/1795220589 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1520005590 con 0x7f15280b3e30 2024-09-17T03:13:49.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.726+0000 7f150f7fe640 1 -- 172.21.15.87:0/1795220589 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1520012870 con 0x7f15280b3e30 2024-09-17T03:13:49.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.726+0000 7fd08fdff640 1 -- 172.21.15.87:0/2852748948 shutdown_connections 2024-09-17T03:13:49.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.726+0000 7fd08fdff640 1 --2- 172.21.15.87:0/2852748948 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd074005350 0x7fd07400b9e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.726+0000 7fd08fdff640 1 -- 172.21.15.87:0/2852748948 >> 172.21.15.87:0/2852748948 conn(0x7fd08001d9d0 msgr2=0x7fd080039690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.729+0000 7f152d389640 1 -- 172.21.15.87:0/1795220589 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1508005290 con 0x7f15280b3e30 2024-09-17T03:13:49.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.726+0000 7fd08fdff640 1 -- 172.21.15.87:0/2852748948 shutdown_connections 2024-09-17T03:13:49.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.726+0000 7fd08fdff640 1 -- 172.21.15.87:0/2852748948 wait complete. 2024-09-17T03:13:49.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.728+0000 7f90a6ffd640 1 -- 172.21.15.87:0/3864808423 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v11)=0 v11) ==== 122+0+0 (secure 0 0 0) 0x7f90a801dc60 con 0x7f90b806ed60 2024-09-17T03:13:49.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.729+0000 7f90a4ff9640 1 -- 172.21.15.87:0/3864808423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90b806ed60 msgr2=0x7f90b814ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.729+0000 7f90a4ff9640 1 --2- 172.21.15.87:0/3864808423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90b806ed60 0x7f90b814ff90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f90a800b780 tx=0x7f90a800bc40 comp rx=0 tx=0).stop 2024-09-17T03:13:49.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.730+0000 7f90a4ff9640 1 -- 172.21.15.87:0/3864808423 shutdown_connections 2024-09-17T03:13:49.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.730+0000 7f90a4ff9640 1 --2- 172.21.15.87:0/3864808423 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f90b8070e90 0x7f90b8154590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.730+0000 7f90a4ff9640 1 --2- 172.21.15.87:0/3864808423 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f90b806f170 0x7f90b8154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.730+0000 7f90a4ff9640 1 --2- 172.21.15.87:0/3864808423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90b806ed60 0x7f90b814ff90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.730+0000 7f90a4ff9640 1 -- 172.21.15.87:0/3864808423 >> 172.21.15.87:0/3864808423 conn(0x7f90b80b1c40 msgr2=0x7f90b809ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.730+0000 7f90a4ff9640 1 -- 172.21.15.87:0/3864808423 shutdown_connections 2024-09-17T03:13:49.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.730+0000 7f90a4ff9640 1 -- 172.21.15.87:0/3864808423 wait complete. 2024-09-17T03:13:49.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.726+0000 7fd3829ed640 1 -- 172.21.15.87:0/2234039483 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd360005290 con 0x7fd37c16b430 2024-09-17T03:13:49.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.730+0000 7fd378ff9640 1 -- 172.21.15.87:0/2234039483 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd37c17cfe0 con 0x7fd37c16b430 2024-09-17T03:13:49.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.734+0000 7f150f7fe640 1 -- 172.21.15.87:0/1795220589 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f1520012ac0 con 0x7f15280b3e30 2024-09-17T03:13:49.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.729+0000 7fd08fdff640 1 Processor -- start 2024-09-17T03:13:49.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.729+0000 7fd08fdff640 1 -- start start 2024-09-17T03:13:49.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.732+0000 7fd08fdff640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd074005350 0x7fd08017f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.732+0000 7fd08fdff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd080022080 0x7fd0800e2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.732+0000 7fd08fdff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0800527a0 0x7fd080052bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:49.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.732+0000 7fd08fdff640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd080005e10 con 0x7fd0800527a0 2024-09-17T03:13:49.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.732+0000 7fd08fdff640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd080006970 con 0x7fd080022080 2024-09-17T03:13:49.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.732+0000 7fd08fdff640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd080005f90 con 0x7fd074005350 2024-09-17T03:13:49.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.732+0000 7fd08d39a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd080022080 0x7fd0800e2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.732+0000 7fd08d39a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd080022080 0x7fd0800e2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.87:33694/0 (socket says 172.21.15.87:33694) 2024-09-17T03:13:49.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.732+0000 7fd08d39a640 1 -- 172.21.15.87:0/3138440327 learned_addr learned my addr 172.21.15.87:0/3138440327 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:13:49.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.734+0000 7fd08db9b640 1 --2- 172.21.15.87:0/3138440327 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd074005350 0x7fd08017f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:49.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.734+0000 7fd08d39a640 1 -- 172.21.15.87:0/3138440327 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd074005350 msgr2=0x7fd08017f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.734+0000 7fd08d39a640 1 --2- 172.21.15.87:0/3138440327 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd074005350 0x7fd08017f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.734+0000 7fd08d39a640 1 -- 172.21.15.87:0/3138440327 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0800527a0 msgr2=0x7fd080052bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.734+0000 7fd08d39a640 1 --2- 172.21.15.87:0/3138440327 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0800527a0 0x7fd080052bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.734+0000 7fd08d39a640 1 -- 172.21.15.87:0/3138440327 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0800530f0 con 0x7fd080022080 2024-09-17T03:13:49.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.742+0000 7fd08d39a640 1 --2- 172.21.15.87:0/3138440327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd080022080 0x7fd0800e2b80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd08806ab90 tx=0x7fd08806b2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:49.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.747+0000 7fd07e7fc640 1 -- 172.21.15.87:0/3138440327 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd088090070 con 0x7fd080022080 2024-09-17T03:13:49.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.747+0000 7fd08fdff640 1 -- 172.21.15.87:0/3138440327 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0800e36b0 con 0x7fd080022080 2024-09-17T03:13:49.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.747+0000 7fd08fdff640 1 -- 172.21.15.87:0/3138440327 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd08004c8e0 con 0x7fd080022080 2024-09-17T03:13:49.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.748+0000 7fd07e7fc640 1 -- 172.21.15.87:0/3138440327 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd088072980 con 0x7fd080022080 2024-09-17T03:13:49.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.748+0000 7fd07e7fc640 1 -- 172.21.15.87:0/3138440327 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd088072dd0 con 0x7fd080022080 2024-09-17T03:13:49.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.750+0000 7fd073fff640 1 -- 172.21.15.87:0/3138440327 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd068005290 con 0x7fd080022080 2024-09-17T03:13:49.752 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi087') 2024-09-17T03:13:49.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.754+0000 7fd07e7fc640 1 -- 172.21.15.87:0/3138440327 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd088093070 con 0x7fd080022080 2024-09-17T03:13:49.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.754+0000 7fd07e7fc640 1 -- 172.21.15.87:0/3138440327 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd08809e3a0 con 0x7fd080022080 2024-09-17T03:13:49.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.758+0000 7fd07e7fc640 1 -- 172.21.15.87:0/3138440327 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd0800530f0 con 0x7fd080022080 2024-09-17T03:13:49.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.791+0000 7f2da5ffb640 1 -- 172.21.15.87:0/1117947764 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f2da0005850 con 0x7f2dc016b6d0 2024-09-17T03:13:49.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.796+0000 7f2da7fff640 1 -- 172.21.15.87:0/1117947764 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v12)=0 v12) ==== 121+0+0 (secure 0 0 0) 0x7f2db8003bf0 con 0x7f2dc016b6d0 2024-09-17T03:13:49.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.797+0000 7f2da5ffb640 1 -- 172.21.15.87:0/1117947764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2dc016b6d0 msgr2=0x7f2dc0216280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.797+0000 7f2da5ffb640 1 --2- 172.21.15.87:0/1117947764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2dc016b6d0 0x7f2dc0216280 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2db800cb60 tx=0x7f2db800a430 comp rx=0 tx=0).stop 2024-09-17T03:13:49.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.797+0000 7f2da5ffb640 1 -- 172.21.15.87:0/1117947764 shutdown_connections 2024-09-17T03:13:49.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.797+0000 7f2da5ffb640 1 --2- 172.21.15.87:0/1117947764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2dc02167c0 0x7f2dc0178290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.797+0000 7f2da5ffb640 1 --2- 172.21.15.87:0/1117947764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2dc016b6d0 0x7f2dc0216280 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.797+0000 7f2da5ffb640 1 --2- 172.21.15.87:0/1117947764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2dc00cee70 0x7f2dc0215950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.797+0000 7f2da5ffb640 1 -- 172.21.15.87:0/1117947764 >> 172.21.15.87:0/1117947764 conn(0x7f2dc00b1b60 msgr2=0x7f2dc016e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.798+0000 7f2da5ffb640 1 -- 172.21.15.87:0/1117947764 shutdown_connections 2024-09-17T03:13:49.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.798+0000 7f2da5ffb640 1 -- 172.21.15.87:0/1117947764 wait complete. 2024-09-17T03:13:49.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.804+0000 7f3a0d7fa640 1 -- 172.21.15.87:0/1151270359 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f3a08005580 con 0x7f3a28205e80 2024-09-17T03:13:49.812 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi087') 2024-09-17T03:13:49.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.817+0000 7f3a0f7fe640 1 -- 172.21.15.87:0/1151270359 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v13)=0 v13) ==== 119+0+0 (secure 0 0 0) 0x7f3a28208b90 con 0x7f3a28205e80 2024-09-17T03:13:49.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.819+0000 7f3a0d7fa640 1 -- 172.21.15.87:0/1151270359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a28205e80 msgr2=0x7f3a2809c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.819+0000 7f3a0d7fa640 1 --2- 172.21.15.87:0/1151270359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a28205e80 0x7f3a2809c480 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3a18000c00 tx=0x7f3a180038f0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.823+0000 7f3a0d7fa640 1 -- 172.21.15.87:0/1151270359 shutdown_connections 2024-09-17T03:13:49.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.823+0000 7f3a0d7fa640 1 --2- 172.21.15.87:0/1151270359 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3a28208240 0x7f3a28208650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.823+0000 7f3a0d7fa640 1 --2- 172.21.15.87:0/1151270359 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3a280c4dd0 0x7f3a282056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.823+0000 7f3a0d7fa640 1 --2- 172.21.15.87:0/1151270359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a28205e80 0x7f3a2809c480 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.823+0000 7f3a0d7fa640 1 -- 172.21.15.87:0/1151270359 >> 172.21.15.87:0/1151270359 conn(0x7f3a280b1b60 msgr2=0x7f3a280aff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.823+0000 7f3a0d7fa640 1 -- 172.21.15.87:0/1151270359 shutdown_connections 2024-09-17T03:13:49.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.823+0000 7f3a0d7fa640 1 -- 172.21.15.87:0/1151270359 wait complete. 2024-09-17T03:13:49.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.824+0000 7fb1181c3640 1 -- 172.21.15.87:0/2982562726 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7fb0f8005580 con 0x7fb11016b6d0 2024-09-17T03:13:49.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.834+0000 7fb1067fc640 1 -- 172.21.15.87:0/2982562726 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v14)=0 v14) ==== 119+0+0 (secure 0 0 0) 0x7fb1080052b0 con 0x7fb11016b6d0 2024-09-17T03:13:49.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.828+0000 7f5bc9ffb640 1 -- 172.21.15.87:0/796738449 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f5be40b4e10 con 0x7f5be415e3d0 2024-09-17T03:13:49.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.836+0000 7fb0f3fff640 1 -- 172.21.15.87:0/2982562726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb11016b6d0 msgr2=0x7fb11009d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.836+0000 7fb0f3fff640 1 --2- 172.21.15.87:0/2982562726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb11016b6d0 0x7fb11009d130 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb10800c7b0 tx=0x7fb10800cc70 comp rx=0 tx=0).stop 2024-09-17T03:13:49.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.836+0000 7fb0f3fff640 1 -- 172.21.15.87:0/2982562726 shutdown_connections 2024-09-17T03:13:49.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.836+0000 7fb0f3fff640 1 --2- 172.21.15.87:0/2982562726 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb11009d670 0x7fb1100b04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.836+0000 7fb0f3fff640 1 --2- 172.21.15.87:0/2982562726 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb11016b6d0 0x7fb11009d130 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb10800c7b0 tx=0x7fb10800cc70 comp rx=0 tx=0).stop 2024-09-17T03:13:49.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.836+0000 7fb0f3fff640 1 --2- 172.21.15.87:0/2982562726 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb1100cee70 0x7fb1100c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.836+0000 7fb0f3fff640 1 -- 172.21.15.87:0/2982562726 >> 172.21.15.87:0/2982562726 conn(0x7fb1100b1b60 msgr2=0x7fb1100aee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.839+0000 7f5bcbfff640 1 -- 172.21.15.87:0/796738449 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v15)=0 v15) ==== 120+0+0 (secure 0 0 0) 0x7f5be415ed20 con 0x7f5be415e3d0 2024-09-17T03:13:49.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.840+0000 7fb0f3fff640 1 -- 172.21.15.87:0/2982562726 shutdown_connections 2024-09-17T03:13:49.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.840+0000 7fb0f3fff640 1 -- 172.21.15.87:0/2982562726 wait complete. 2024-09-17T03:13:49.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.841+0000 7f5bc9ffb640 1 -- 172.21.15.87:0/796738449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5be415e3d0 msgr2=0x7f5be415e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.841+0000 7f5bc9ffb640 1 --2- 172.21.15.87:0/796738449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5be415e3d0 0x7f5be415e7e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800c500 tx=0x7f5bd800c9c0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.841+0000 7f5bc9ffb640 1 -- 172.21.15.87:0/796738449 shutdown_connections 2024-09-17T03:13:49.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.841+0000 7f5bc9ffb640 1 --2- 172.21.15.87:0/796738449 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5be415e3d0 0x7f5be415e7e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.841+0000 7f5bc9ffb640 1 --2- 172.21.15.87:0/796738449 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5be40ace40 0x7f5be415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.841+0000 7f5bc9ffb640 1 --2- 172.21.15.87:0/796738449 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5be406fa80 0x7f5be4148b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.841+0000 7f5bc9ffb640 1 -- 172.21.15.87:0/796738449 >> 172.21.15.87:0/796738449 conn(0x7f5be40b1b60 msgr2=0x7f5be40cfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.853+0000 7f5bc9ffb640 1 -- 172.21.15.87:0/796738449 shutdown_connections 2024-09-17T03:13:49.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.853+0000 7f5bc9ffb640 1 -- 172.21.15.87:0/796738449 wait complete. 2024-09-17T03:13:49.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.854+0000 7f2ce9ffb640 1 -- 172.21.15.87:0/3259846363 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f2ce4005580 con 0x7f2d0409c650 2024-09-17T03:13:49.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.859+0000 7f2cebfff640 1 -- 172.21.15.87:0/3259846363 <== mon.2 v2:172.21.15.149:3301/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v16)=0 v16) ==== 121+0+0 (secure 0 0 0) 0x7f2d0409cfa0 con 0x7f2d0409c650 2024-09-17T03:13:49.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.869+0000 7f2ce9ffb640 1 -- 172.21.15.87:0/3259846363 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2d0409c650 msgr2=0x7f2d0409ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.869+0000 7f2ce9ffb640 1 --2- 172.21.15.87:0/3259846363 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2d0409c650 0x7f2d0409ca60 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800b410 tx=0x7f2cf800b8d0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.869 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi087') 2024-09-17T03:13:49.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.872+0000 7f2ce9ffb640 1 -- 172.21.15.87:0/3259846363 shutdown_connections 2024-09-17T03:13:49.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.872+0000 7f2ce9ffb640 1 --2- 172.21.15.87:0/3259846363 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2d0409c650 0x7f2d0409ca60 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.872+0000 7f2ce9ffb640 1 --2- 172.21.15.87:0/3259846363 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2d0416b6d0 0x7f2d04205040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.872+0000 7f2ce9ffb640 1 --2- 172.21.15.87:0/3259846363 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d04073540 0x7f2d040c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.872+0000 7f2ce9ffb640 1 -- 172.21.15.87:0/3259846363 >> 172.21.15.87:0/3259846363 conn(0x7f2d040b1b60 msgr2=0x7f2d0416bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.872+0000 7f2ce9ffb640 1 -- 172.21.15.87:0/3259846363 shutdown_connections 2024-09-17T03:13:49.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.872+0000 7f2ce9ffb640 1 -- 172.21.15.87:0/3259846363 wait complete. 2024-09-17T03:13:49.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.875+0000 7f65c2263640 1 -- 172.21.15.87:0/1980948656 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f659c005580 con 0x7f65bc162710 2024-09-17T03:13:49.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.883+0000 7f65b8ff9640 1 -- 172.21.15.87:0/1980948656 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v17)=0 v17) ==== 120+0+0 (secure 0 0 0) 0x7f65b000fdb0 con 0x7f65bc162710 2024-09-17T03:13:49.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.887+0000 7f0898ff9640 1 -- 172.21.15.87:0/2274944749 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f0884005850 con 0x7f08a40d8580 2024-09-17T03:13:49.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.888+0000 7f65a27fc640 1 -- 172.21.15.87:0/1980948656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65bc162710 msgr2=0x7f65bc178380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.888+0000 7f65a27fc640 1 --2- 172.21.15.87:0/1980948656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65bc162710 0x7f65bc178380 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f65b000c9e0 tx=0x7f65b000cea0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.888+0000 7f65a27fc640 1 -- 172.21.15.87:0/1980948656 shutdown_connections 2024-09-17T03:13:49.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.888+0000 7f65a27fc640 1 --2- 172.21.15.87:0/1980948656 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65bc162710 0x7f65bc178380 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.888+0000 7f65a27fc640 1 --2- 172.21.15.87:0/1980948656 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f65bc16b6d0 0x7f65bc177e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.888+0000 7f65a27fc640 1 --2- 172.21.15.87:0/1980948656 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65bc0cee70 0x7f65bc15fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.888+0000 7f65a27fc640 1 -- 172.21.15.87:0/1980948656 >> 172.21.15.87:0/1980948656 conn(0x7f65bc0b1b60 msgr2=0x7f65bc16e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.888+0000 7f65a27fc640 1 -- 172.21.15.87:0/1980948656 shutdown_connections 2024-09-17T03:13:49.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.889+0000 7f65a27fc640 1 -- 172.21.15.87:0/1980948656 wait complete. 2024-09-17T03:13:49.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.893+0000 7f089affd640 1 -- 172.21.15.87:0/2274944749 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v18)=0 v18) ==== 106+0+0 (secure 0 0 0) 0x7f089c021070 con 0x7f08a40d8580 2024-09-17T03:13:49.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.896+0000 7f0898ff9640 1 -- 172.21.15.87:0/2274944749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08a40d8580 msgr2=0x7f08a4034d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.896+0000 7f0898ff9640 1 --2- 172.21.15.87:0/2274944749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08a40d8580 0x7f08a4034d70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f08a40e2be0 tx=0x7f089c009670 comp rx=0 tx=0).stop 2024-09-17T03:13:49.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.896+0000 7f0898ff9640 1 -- 172.21.15.87:0/2274944749 shutdown_connections 2024-09-17T03:13:49.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.896+0000 7f0898ff9640 1 --2- 172.21.15.87:0/2274944749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08a40d8580 0x7f08a4034d70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.896+0000 7f0898ff9640 1 --2- 172.21.15.87:0/2274944749 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f08a40d8030 0x7f08a4034830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.896+0000 7f0898ff9640 1 --2- 172.21.15.87:0/2274944749 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0894003a30 0x7f08a40d7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.896+0000 7f0898ff9640 1 -- 172.21.15.87:0/2274944749 >> 172.21.15.87:0/2274944749 conn(0x7f08a4051450 msgr2=0x7f08a4052750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.896+0000 7f0898ff9640 1 -- 172.21.15.87:0/2274944749 shutdown_connections 2024-09-17T03:13:49.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.896+0000 7f0898ff9640 1 -- 172.21.15.87:0/2274944749 wait complete. 2024-09-17T03:13:49.899 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi087') 2024-09-17T03:13:49.899 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi087') 2024-09-17T03:13:49.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.902+0000 7efe20ff9640 1 -- 172.21.15.87:0/4258095832 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7efe18005580 con 0x7efe3415b2d0 2024-09-17T03:13:49.908 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi087') 2024-09-17T03:13:49.908 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi087') 2024-09-17T03:13:49.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.909+0000 7efe22ffd640 1 -- 172.21.15.87:0/4258095832 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v19)=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7efe2403cd20 con 0x7efe3415b2d0 2024-09-17T03:13:49.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.912+0000 7efe20ff9640 1 -- 172.21.15.87:0/4258095832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe3415b2d0 msgr2=0x7efe34159860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.912+0000 7efe20ff9640 1 --2- 172.21.15.87:0/4258095832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe3415b2d0 0x7efe34159860 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efe24000c00 tx=0x7efe24033ba0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.912+0000 7efe20ff9640 1 -- 172.21.15.87:0/4258095832 shutdown_connections 2024-09-17T03:13:49.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.912+0000 7efe20ff9640 1 --2- 172.21.15.87:0/4258095832 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efe3415b6e0 0x7efe34159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.912+0000 7efe20ff9640 1 --2- 172.21.15.87:0/4258095832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe3415b2d0 0x7efe34159860 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.912+0000 7efe20ff9640 1 --2- 172.21.15.87:0/4258095832 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efe340cca40 0x7efe34201bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.912+0000 7efe20ff9640 1 -- 172.21.15.87:0/4258095832 >> 172.21.15.87:0/4258095832 conn(0x7efe340d0750 msgr2=0x7efe340cd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.912+0000 7efe20ff9640 1 -- 172.21.15.87:0/4258095832 shutdown_connections 2024-09-17T03:13:49.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.913+0000 7efe20ff9640 1 -- 172.21.15.87:0/4258095832 wait complete. 2024-09-17T03:13:49.921 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.922+0000 7f05bb920640 1 -- 172.21.15.87:0/3502816569 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f05b40d0620 con 0x7f05b40cc350 2024-09-17T03:13:49.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.925+0000 7fd3829ed640 1 -- 172.21.15.87:0/2234039483 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fd360005580 con 0x7fd37c16b430 2024-09-17T03:13:49.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.926+0000 7f05a9ffb640 1 -- 172.21.15.87:0/3502816569 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v20)=0 v20) ==== 111+0+0 (secure 0 0 0) 0x7f05b00193b0 con 0x7f05b40cc350 2024-09-17T03:13:49.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.927+0000 7f05bb920640 1 -- 172.21.15.87:0/3502816569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05b40cc350 msgr2=0x7f05b4158560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.927+0000 7f05bb920640 1 --2- 172.21.15.87:0/3502816569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05b40cc350 0x7f05b4158560 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f05b000b440 tx=0x7f05b000b900 comp rx=0 tx=0).stop 2024-09-17T03:13:49.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.927+0000 7f05bb920640 1 -- 172.21.15.87:0/3502816569 shutdown_connections 2024-09-17T03:13:49.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.927+0000 7f05bb920640 1 --2- 172.21.15.87:0/3502816569 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f05b415e220 0x7f05b4152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.927+0000 7f05bb920640 1 --2- 172.21.15.87:0/3502816569 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05b40cc350 0x7f05b4158560 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.927+0000 7f05bb920640 1 --2- 172.21.15.87:0/3502816569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05b40ca580 0x7f05b41575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.927+0000 7f05bb920640 1 -- 172.21.15.87:0/3502816569 >> 172.21.15.87:0/3502816569 conn(0x7f05b40cff30 msgr2=0x7f05b40afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.927+0000 7f05bb920640 1 -- 172.21.15.87:0/3502816569 shutdown_connections 2024-09-17T03:13:49.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.927+0000 7f05bb920640 1 -- 172.21.15.87:0/3502816569 wait complete. 2024-09-17T03:13:49.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.929+0000 7fd378ff9640 1 -- 172.21.15.87:0/2234039483 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v21)=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7fd37403ad00 con 0x7fd37c16b430 2024-09-17T03:13:49.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.930+0000 7fd3829ed640 1 -- 172.21.15.87:0/2234039483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd37c16b430 msgr2=0x7fd37c15c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.930+0000 7fd3829ed640 1 --2- 172.21.15.87:0/2234039483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd37c16b430 0x7fd37c15c120 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd374009d60 tx=0x7fd374003f10 comp rx=0 tx=0).stop 2024-09-17T03:13:49.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.931+0000 7fd3829ed640 1 -- 172.21.15.87:0/2234039483 shutdown_connections 2024-09-17T03:13:49.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.931+0000 7fd3829ed640 1 --2- 172.21.15.87:0/2234039483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd37c16b430 0x7fd37c15c120 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.931+0000 7fd3829ed640 1 --2- 172.21.15.87:0/2234039483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd37c0b3b70 0x7fd37c15bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.931+0000 7fd3829ed640 1 --2- 172.21.15.87:0/2234039483 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd37c08b0b0 0x7fd37c17caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.931+0000 7fd3829ed640 1 -- 172.21.15.87:0/2234039483 >> 172.21.15.87:0/2234039483 conn(0x7fd37c0b1c40 msgr2=0x7fd37c15d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.931+0000 7fd3829ed640 1 -- 172.21.15.87:0/2234039483 shutdown_connections 2024-09-17T03:13:49.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.931+0000 7fd3829ed640 1 -- 172.21.15.87:0/2234039483 wait complete. 2024-09-17T03:13:49.939 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi087') 2024-09-17T03:13:49.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.959+0000 7fd073fff640 1 -- 172.21.15.87:0/3138440327 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7fd068005850 con 0x7fd080022080 2024-09-17T03:13:49.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.962+0000 7f152d389640 1 -- 172.21.15.87:0/1795220589 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f1508005580 con 0x7f15280b3e30 2024-09-17T03:13:49.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.963+0000 7fd07e7fc640 1 -- 172.21.15.87:0/3138440327 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v22)=0 v22) ==== 108+0+0 (secure 0 0 0) 0x7fd08806a120 con 0x7fd080022080 2024-09-17T03:13:49.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.964+0000 7fd073fff640 1 -- 172.21.15.87:0/3138440327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd080022080 msgr2=0x7fd0800e2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.964+0000 7fd073fff640 1 --2- 172.21.15.87:0/3138440327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd080022080 0x7fd0800e2b80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd08806ab90 tx=0x7fd08806b2b0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.964+0000 7fd073fff640 1 -- 172.21.15.87:0/3138440327 shutdown_connections 2024-09-17T03:13:49.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.964+0000 7fd073fff640 1 --2- 172.21.15.87:0/3138440327 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0800527a0 0x7fd080052bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.964+0000 7fd073fff640 1 --2- 172.21.15.87:0/3138440327 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd080022080 0x7fd0800e2b80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.964+0000 7fd073fff640 1 --2- 172.21.15.87:0/3138440327 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd074005350 0x7fd08017f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.964+0000 7fd073fff640 1 -- 172.21.15.87:0/3138440327 >> 172.21.15.87:0/3138440327 conn(0x7fd08001d9d0 msgr2=0x7fd0800054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.964+0000 7fd073fff640 1 -- 172.21.15.87:0/3138440327 shutdown_connections 2024-09-17T03:13:49.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.964+0000 7fd073fff640 1 -- 172.21.15.87:0/3138440327 wait complete. 2024-09-17T03:13:49.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.965+0000 7f150f7fe640 1 -- 172.21.15.87:0/1795220589 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v23)=0 v23) ==== 125+0+0 (secure 0 0 0) 0x7f1520012ca0 con 0x7f15280b3e30 2024-09-17T03:13:49.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.967+0000 7f152d389640 1 -- 172.21.15.87:0/1795220589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15280b3e30 msgr2=0x7f15281621f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:49.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.967+0000 7f152d389640 1 --2- 172.21.15.87:0/1795220589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15280b3e30 0x7f15281621f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f152000c6c0 tx=0x7f152000cb80 comp rx=0 tx=0).stop 2024-09-17T03:13:49.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.967+0000 7f152d389640 1 -- 172.21.15.87:0/1795220589 shutdown_connections 2024-09-17T03:13:49.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.967+0000 7f152d389640 1 --2- 172.21.15.87:0/1795220589 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f152815ef00 0x7f1528203f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.967+0000 7f152d389640 1 --2- 172.21.15.87:0/1795220589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15280b3e30 0x7f15281621f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.967+0000 7f152d389640 1 --2- 172.21.15.87:0/1795220589 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f15280b1310 0x7f1528161cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:49.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.967+0000 7f152d389640 1 -- 172.21.15.87:0/1795220589 >> 172.21.15.87:0/1795220589 conn(0x7f15280b3310 msgr2=0x7f15280b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:49.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.967+0000 7f152d389640 1 -- 172.21.15.87:0/1795220589 shutdown_connections 2024-09-17T03:13:49.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:49.967+0000 7f152d389640 1 -- 172.21.15.87:0/1795220589 wait complete. 2024-09-17T03:13:49.976 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi087') 2024-09-17T03:13:49.976 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi087') 2024-09-17T03:13:49.981 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi087') 2024-09-17T03:13:49.981 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi087') 2024-09-17T03:13:49.981 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config dump 2024-09-17T03:13:50.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.133+0000 7f0842230640 1 -- 172.21.15.87:0/1545376501 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0828004800 msgr2=0x7f0828004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:50.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.133+0000 7f0842230640 1 --2- 172.21.15.87:0/1545376501 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0828004800 0x7f0828004c10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f082400c290 tx=0x7f0828005830 comp rx=0 tx=0).stop 2024-09-17T03:13:50.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.133+0000 7f083bfff640 1 -- 172.21.15.87:0/1545376501 reap_dead start 2024-09-17T03:13:50.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.133+0000 7f0842230640 1 -- 172.21.15.87:0/1545376501 shutdown_connections 2024-09-17T03:13:50.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.133+0000 7f0842230640 1 -- 172.21.15.87:0/1545376501 >> 172.21.15.87:0/1545376501 conn(0x7f083c0b2d10 msgr2=0x7f083c0b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:50.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.133+0000 7f0842230640 1 -- 172.21.15.87:0/1545376501 shutdown_connections 2024-09-17T03:13:50.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.133+0000 7f0842230640 1 -- 172.21.15.87:0/1545376501 wait complete. 2024-09-17T03:13:50.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.135+0000 7f0842230640 1 Processor -- start 2024-09-17T03:13:50.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.135+0000 7f0842230640 1 -- start start 2024-09-17T03:13:50.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.136+0000 7f0842230640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0828004800 0x7f083c14ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:50.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.136+0000 7f0842230640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f083c14bb90 0x7f083c143920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:50.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.136+0000 7f0842230640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f083c150a00 0x7f083c150e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:50.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.136+0000 7f0842230640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f083c15fa10 con 0x7f0828004800 2024-09-17T03:13:50.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.136+0000 7f0842230640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f083c15f890 con 0x7f083c14bb90 2024-09-17T03:13:50.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.136+0000 7f0842230640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f083c15bb90 con 0x7f083c150a00 2024-09-17T03:13:50.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.137+0000 7f083b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0828004800 0x7f083c14ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:50.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.137+0000 7f083b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0828004800 0x7f083c14ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41090/0 (socket says 172.21.15.87:41090) 2024-09-17T03:13:50.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.137+0000 7f083b7fe640 1 -- 172.21.15.87:0/3043212301 learned_addr learned my addr 172.21.15.87:0/3043212301 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:13:50.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.137+0000 7f083bfff640 1 --2- 172.21.15.87:0/3043212301 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f083c150a00 0x7f083c150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:50.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.137+0000 7f083affd640 1 --2- 172.21.15.87:0/3043212301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f083c14bb90 0x7f083c143920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:50.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.137+0000 7f083b7fe640 1 -- 172.21.15.87:0/3043212301 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f083c150a00 msgr2=0x7f083c150e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:50.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.137+0000 7f083b7fe640 1 --2- 172.21.15.87:0/3043212301 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f083c150a00 0x7f083c150e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:50.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.137+0000 7f083b7fe640 1 -- 172.21.15.87:0/3043212301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f083c14bb90 msgr2=0x7f083c143920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:50.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.137+0000 7f083b7fe640 1 --2- 172.21.15.87:0/3043212301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f083c14bb90 0x7f083c143920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:50.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.137+0000 7f083b7fe640 1 -- 172.21.15.87:0/3043212301 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083c151350 con 0x7f0828004800 2024-09-17T03:13:50.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.137+0000 7f083affd640 1 --2- 172.21.15.87:0/3043212301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f083c14bb90 0x7f083c143920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:13:50.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.137+0000 7f083b7fe640 1 --2- 172.21.15.87:0/3043212301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0828004800 0x7f083c14ef50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f082c005d00 tx=0x7f082c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:50.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.137+0000 7f0833fff640 1 -- 172.21.15.87:0/3043212301 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c01e070 con 0x7f0828004800 2024-09-17T03:13:50.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.137+0000 7f0833fff640 1 -- 172.21.15.87:0/3043212301 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f082c00e7c0 con 0x7f0828004800 2024-09-17T03:13:50.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.137+0000 7f0842230640 1 -- 172.21.15.87:0/3043212301 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f083c1576d0 con 0x7f0828004800 2024-09-17T03:13:50.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.137+0000 7f0833fff640 1 -- 172.21.15.87:0/3043212301 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c00ebf0 con 0x7f0828004800 2024-09-17T03:13:50.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.137+0000 7f0842230640 1 -- 172.21.15.87:0/3043212301 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f083c157c90 con 0x7f0828004800 2024-09-17T03:13:50.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.138+0000 7f0833fff640 1 -- 172.21.15.87:0/3043212301 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f082c021070 con 0x7f0828004800 2024-09-17T03:13:50.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.138+0000 7f0833fff640 1 -- 172.21.15.87:0/3043212301 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f082c019770 con 0x7f0828004800 2024-09-17T03:13:50.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.139+0000 7f0842230640 1 -- 172.21.15.87:0/3043212301 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f083c0b0710 con 0x7f0828004800 2024-09-17T03:13:50.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.141+0000 7f0833fff640 1 -- 172.21.15.87:0/3043212301 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f083c151350 con 0x7f0828004800 2024-09-17T03:13:50.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.200+0000 7f0842230640 1 -- 172.21.15.87:0/3043212301 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f083c15af00 con 0x7f0828004800 2024-09-17T03:13:50.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.200+0000 7f0833fff640 1 -- 172.21.15.87:0/3043212301 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v23) ==== 59+0+1472 (secure 0 0 0) 0x7f082c00edf0 con 0x7f0828004800 2024-09-17T03:13:50.199 INFO:teuthology.orchestra.run.smithi087.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T03:13:50.199 INFO:teuthology.orchestra.run.smithi087.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T03:13:50.199 INFO:teuthology.orchestra.run.smithi087.stdout:mgr advanced client_mount_timeout 30 2024-09-17T03:13:50.199 INFO:teuthology.orchestra.run.smithi087.stdout:mgr advanced debug_client 20/20 2024-09-17T03:13:50.199 INFO:teuthology.orchestra.run.smithi087.stdout:mgr advanced debug_mgr 20/20 2024-09-17T03:13:50.199 INFO:teuthology.orchestra.run.smithi087.stdout:mgr advanced debug_ms 1/1 2024-09-17T03:13:50.200 INFO:teuthology.orchestra.run.smithi087.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T03:13:50.200 INFO:teuthology.orchestra.run.smithi087.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T03:13:50.200 INFO:teuthology.orchestra.run.smithi087.stdout:mds advanced debug_mds 20/20 2024-09-17T03:13:50.200 INFO:teuthology.orchestra.run.smithi087.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T03:13:50.200 INFO:teuthology.orchestra.run.smithi087.stdout:mds advanced debug_ms 2/2 2024-09-17T03:13:50.200 INFO:teuthology.orchestra.run.smithi087.stdout:mds dev mds_debug_frag true 2024-09-17T03:13:50.200 INFO:teuthology.orchestra.run.smithi087.stdout:mds dev mds_debug_scatterstat true 2024-09-17T03:13:50.200 INFO:teuthology.orchestra.run.smithi087.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T03:13:50.200 INFO:teuthology.orchestra.run.smithi087.stdout:mds dev mds_verify_scatter true 2024-09-17T03:13:50.200 INFO:teuthology.orchestra.run.smithi087.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T03:13:50.200 INFO:teuthology.orchestra.run.smithi087.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T03:13:50.200 INFO:teuthology.orchestra.run.smithi087.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T03:13:50.200 INFO:teuthology.orchestra.run.smithi087.stdout:client advanced client_mount_timeout 600 2024-09-17T03:13:50.200 INFO:teuthology.orchestra.run.smithi087.stdout:client advanced debug_client 20/20 2024-09-17T03:13:50.201 INFO:teuthology.orchestra.run.smithi087.stdout:client advanced debug_ms 1/1 2024-09-17T03:13:50.201 INFO:teuthology.orchestra.run.smithi087.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T03:13:50.201 INFO:teuthology.orchestra.run.smithi087.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T03:13:50.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.201+0000 7f0842230640 1 -- 172.21.15.87:0/3043212301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0828004800 msgr2=0x7f083c14ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:50.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.201+0000 7f0842230640 1 --2- 172.21.15.87:0/3043212301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0828004800 0x7f083c14ef50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f082c005d00 tx=0x7f082c00f790 comp rx=0 tx=0).stop 2024-09-17T03:13:50.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.201+0000 7f0842230640 1 -- 172.21.15.87:0/3043212301 shutdown_connections 2024-09-17T03:13:50.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.201+0000 7f0842230640 1 --2- 172.21.15.87:0/3043212301 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f083c150a00 0x7f083c150e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:50.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.201+0000 7f0842230640 1 --2- 172.21.15.87:0/3043212301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f083c14bb90 0x7f083c143920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:50.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.201+0000 7f0842230640 1 --2- 172.21.15.87:0/3043212301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0828004800 0x7f083c14ef50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:50.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.201+0000 7f0842230640 1 -- 172.21.15.87:0/3043212301 >> 172.21.15.87:0/3043212301 conn(0x7f083c0b2d10 msgr2=0x7f083c0cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:50.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.201+0000 7f0842230640 1 -- 172.21.15.87:0/3043212301 shutdown_connections 2024-09-17T03:13:50.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.201+0000 7f0842230640 1 -- 172.21.15.87:0/3043212301 wait complete. 2024-09-17T03:13:50.211 INFO:tasks.ceph:Setting crush tunables to default 2024-09-17T03:13:50.211 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd crush tunables default 2024-09-17T03:13:50.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.325+0000 7fb0551e4640 1 -- 172.21.15.87:0/1668418679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb03c003c10 msgr2=0x7fb03c004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:50.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.325+0000 7fb0551e4640 1 --2- 172.21.15.87:0/1668418679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb03c003c10 0x7fb03c004060 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb040006f90 tx=0x7fb03c005830 comp rx=0 tx=0).stop 2024-09-17T03:13:50.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.325+0000 7fb04f59e640 1 -- 172.21.15.87:0/1668418679 reap_dead start 2024-09-17T03:13:50.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.325+0000 7fb0551e4640 1 -- 172.21.15.87:0/1668418679 shutdown_connections 2024-09-17T03:13:50.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.325+0000 7fb0551e4640 1 -- 172.21.15.87:0/1668418679 >> 172.21.15.87:0/1668418679 conn(0x7fb05006d440 msgr2=0x7fb0500b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:50.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.325+0000 7fb0551e4640 1 -- 172.21.15.87:0/1668418679 shutdown_connections 2024-09-17T03:13:50.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.325+0000 7fb0551e4640 1 -- 172.21.15.87:0/1668418679 wait complete. 2024-09-17T03:13:50.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.327+0000 7fb0551e4640 1 Processor -- start 2024-09-17T03:13:50.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.327+0000 7fb0551e4640 1 -- start start 2024-09-17T03:13:50.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.328+0000 7fb0551e4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb03c003c10 0x7fb0501f7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:50.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.328+0000 7fb0551e4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0500cc0a0 0x7fb050140c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:50.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.328+0000 7fb0551e4640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb050141160 0x7fb0501f33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:50.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.328+0000 7fb0551e4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb05015d5b0 con 0x7fb0500cc0a0 2024-09-17T03:13:50.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.328+0000 7fb0551e4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb05015d430 con 0x7fb03c003c10 2024-09-17T03:13:50.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.328+0000 7fb0551e4640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb0500ccdb0 con 0x7fb050141160 2024-09-17T03:13:50.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.328+0000 7fb04e59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0500cc0a0 0x7fb050140c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:50.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.328+0000 7fb04e59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0500cc0a0 0x7fb050140c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41096/0 (socket says 172.21.15.87:41096) 2024-09-17T03:13:50.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.328+0000 7fb04e59c640 1 -- 172.21.15.87:0/4178903317 learned_addr learned my addr 172.21.15.87:0/4178903317 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:13:50.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.328+0000 7fb04ed9d640 1 --2- 172.21.15.87:0/4178903317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb03c003c10 0x7fb0501f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:50.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.328+0000 7fb04f59e640 1 --2- 172.21.15.87:0/4178903317 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb050141160 0x7fb0501f33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:50.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.329+0000 7fb04e59c640 1 -- 172.21.15.87:0/4178903317 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb050141160 msgr2=0x7fb0501f33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:50.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.329+0000 7fb04e59c640 1 --2- 172.21.15.87:0/4178903317 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb050141160 0x7fb0501f33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:50.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.329+0000 7fb04e59c640 1 -- 172.21.15.87:0/4178903317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb03c003c10 msgr2=0x7fb0501f7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:50.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.329+0000 7fb04e59c640 1 --2- 172.21.15.87:0/4178903317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb03c003c10 0x7fb0501f7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:50.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.329+0000 7fb04e59c640 1 -- 172.21.15.87:0/4178903317 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05014e5c0 con 0x7fb0500cc0a0 2024-09-17T03:13:50.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.329+0000 7fb04ed9d640 1 --2- 172.21.15.87:0/4178903317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb03c003c10 0x7fb0501f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:13:50.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.329+0000 7fb04e59c640 1 --2- 172.21.15.87:0/4178903317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0500cc0a0 0x7fb050140c20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb038011ae0 tx=0x7fb038009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:50.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.329+0000 7fb0477fe640 1 -- 172.21.15.87:0/4178903317 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03801e070 con 0x7fb0500cc0a0 2024-09-17T03:13:50.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.329+0000 7fb0477fe640 1 -- 172.21.15.87:0/4178903317 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb03800fe70 con 0x7fb0500cc0a0 2024-09-17T03:13:50.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.329+0000 7fb0551e4640 1 -- 172.21.15.87:0/4178903317 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0501f6ad0 con 0x7fb0500cc0a0 2024-09-17T03:13:50.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.329+0000 7fb0477fe640 1 -- 172.21.15.87:0/4178903317 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038004290 con 0x7fb0500cc0a0 2024-09-17T03:13:50.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.329+0000 7fb0551e4640 1 -- 172.21.15.87:0/4178903317 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0501f6d40 con 0x7fb0500cc0a0 2024-09-17T03:13:50.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.329+0000 7fb0477fe640 1 -- 172.21.15.87:0/4178903317 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb038021070 con 0x7fb0500cc0a0 2024-09-17T03:13:50.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.329+0000 7fb0477fe640 1 -- 172.21.15.87:0/4178903317 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb038023a20 con 0x7fb0500cc0a0 2024-09-17T03:13:50.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.330+0000 7fb0551e4640 1 -- 172.21.15.87:0/4178903317 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb050141620 con 0x7fb0500cc0a0 2024-09-17T03:13:50.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.332+0000 7fb0477fe640 1 -- 172.21.15.87:0/4178903317 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fb0380035f0 con 0x7fb0500cc0a0 2024-09-17T03:13:50.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.395+0000 7fb0551e4640 1 -- 172.21.15.87:0/4178903317 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fb0500cd310 con 0x7fb0500cc0a0 2024-09-17T03:13:50.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.396+0000 7fb0477fe640 1 -- 172.21.15.87:0/4178903317 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fb0380037d0 con 0x7fb0500cc0a0 2024-09-17T03:13:50.395 INFO:teuthology.orchestra.run.smithi087.stderr:adjusted tunables profile to default 2024-09-17T03:13:50.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.397+0000 7fb0551e4640 1 -- 172.21.15.87:0/4178903317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0500cc0a0 msgr2=0x7fb050140c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:50.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.397+0000 7fb0551e4640 1 --2- 172.21.15.87:0/4178903317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0500cc0a0 0x7fb050140c20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb038011ae0 tx=0x7fb038009e90 comp rx=0 tx=0).stop 2024-09-17T03:13:50.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.397+0000 7fb0551e4640 1 -- 172.21.15.87:0/4178903317 shutdown_connections 2024-09-17T03:13:50.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.397+0000 7fb0551e4640 1 --2- 172.21.15.87:0/4178903317 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb050141160 0x7fb0501f33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:50.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.397+0000 7fb0551e4640 1 --2- 172.21.15.87:0/4178903317 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0500cc0a0 0x7fb050140c20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:50.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.397+0000 7fb0551e4640 1 --2- 172.21.15.87:0/4178903317 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb03c003c10 0x7fb0501f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:50.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.397+0000 7fb0551e4640 1 -- 172.21.15.87:0/4178903317 >> 172.21.15.87:0/4178903317 conn(0x7fb05006d440 msgr2=0x7fb050075c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:50.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.397+0000 7fb0551e4640 1 -- 172.21.15.87:0/4178903317 shutdown_connections 2024-09-17T03:13:50.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:50.397+0000 7fb0551e4640 1 -- 172.21.15.87:0/4178903317 wait complete. 2024-09-17T03:13:50.407 INFO:tasks.ceph:check_enable_crimson: False 2024-09-17T03:13:50.407 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-17T03:13:50.407 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:13:50.408 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-17T03:13:50.431 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:13:50.431 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-17T03:13:50.496 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:13:50.496 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-17T03:13:50.561 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:13:50.561 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-17T03:13:50.626 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:13:50.626 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-17T03:13:50.653 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:13:50.653 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-17T03:13:50.718 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:13:50.718 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-17T03:13:50.783 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:13:50.784 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-17T03:13:50.848 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --cluster ceph osd new 22b83449-856f-4b82-b290-cd0e9e438bbf 0 2024-09-17T03:13:51.005 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.005+0000 7f7d30427640 1 -- 172.21.15.149:0/3007741732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d2809aa30 msgr2=0x7f7d2815c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:51.005 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.005+0000 7f7d30427640 1 --2- 172.21.15.149:0/3007741732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d2809aa30 0x7f7d2815c660 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7d18005fd0 tx=0x7f7d1802a5d0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.005 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.005+0000 7f7d30427640 1 -- 172.21.15.149:0/3007741732 shutdown_connections 2024-09-17T03:13:51.006 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.005+0000 7f7d30427640 1 --2- 172.21.15.149:0/3007741732 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7d280cef30 0x7f7d280cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.006 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.005+0000 7f7d30427640 1 --2- 172.21.15.149:0/3007741732 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d2809aa30 0x7f7d2815c660 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.006 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.005+0000 7f7d30427640 1 -- 172.21.15.149:0/3007741732 >> 172.21.15.149:0/3007741732 conn(0x7f7d280741b0 msgr2=0x7f7d280d1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:51.006 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.005+0000 7f7d30427640 1 -- 172.21.15.149:0/3007741732 shutdown_connections 2024-09-17T03:13:51.006 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.005+0000 7f7d30427640 1 -- 172.21.15.149:0/3007741732 wait complete. 2024-09-17T03:13:51.006 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.007+0000 7f7d30427640 1 Processor -- start 2024-09-17T03:13:51.007 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.007+0000 7f7d30427640 1 -- start start 2024-09-17T03:13:51.010 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.008+0000 7f7d30427640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d2809aa30 0x7f7d280c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:51.010 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.009+0000 7f7d30427640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7d280cef30 0x7f7d280b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:51.010 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.009+0000 7f7d30427640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d2815f1d0 0x7f7d28153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:51.010 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.009+0000 7f7d30427640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2815e920 con 0x7f7d2815f1d0 2024-09-17T03:13:51.010 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.009+0000 7f7d30427640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7d2815e7a0 con 0x7f7d2809aa30 2024-09-17T03:13:51.010 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.009+0000 7f7d30427640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f7d2815eaa0 con 0x7f7d280cef30 2024-09-17T03:13:51.010 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.009+0000 7f7d2e1c3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d2809aa30 0x7f7d280c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:51.010 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.009+0000 7f7d2d9c2640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7d280cef30 0x7f7d280b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:51.010 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.009+0000 7f7d2d9c2640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7d280cef30 0x7f7d280b1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.149:60008/0 (socket says 172.21.15.149:60008) 2024-09-17T03:13:51.010 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.009+0000 7f7d2d9c2640 1 -- 172.21.15.149:0/4195485872 learned_addr learned my addr 172.21.15.149:0/4195485872 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-09-17T03:13:51.010 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.009+0000 7f7d2e9c4640 1 --2- 172.21.15.149:0/4195485872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d2815f1d0 0x7f7d28153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:51.011 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.009+0000 7f7d2d9c2640 1 -- 172.21.15.149:0/4195485872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d2809aa30 msgr2=0x7f7d280c5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:51.011 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.009+0000 7f7d2d9c2640 1 --2- 172.21.15.149:0/4195485872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d2809aa30 0x7f7d280c5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.011 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.009+0000 7f7d2d9c2640 1 -- 172.21.15.149:0/4195485872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d2815f1d0 msgr2=0x7f7d28153120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:51.011 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.009+0000 7f7d2d9c2640 1 --2- 172.21.15.149:0/4195485872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d2815f1d0 0x7f7d28153120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.011 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.009+0000 7f7d2d9c2640 1 -- 172.21.15.149:0/4195485872 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d280c4fc0 con 0x7f7d280cef30 2024-09-17T03:13:51.011 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.009+0000 7f7d2d9c2640 1 --2- 172.21.15.149:0/4195485872 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7d280cef30 0x7f7d280b1d10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7d240099d0 tx=0x7f7d24009e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:51.011 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.009+0000 7f7d1effd640 1 -- 172.21.15.149:0/4195485872 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2400bab0 con 0x7f7d280cef30 2024-09-17T03:13:51.011 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.009+0000 7f7d1effd640 1 -- 172.21.15.149:0/4195485872 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d2400bc50 con 0x7f7d280cef30 2024-09-17T03:13:51.011 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.009+0000 7f7d1effd640 1 -- 172.21.15.149:0/4195485872 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d24005020 con 0x7f7d280cef30 2024-09-17T03:13:51.011 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.009+0000 7f7d30427640 1 -- 172.21.15.149:0/4195485872 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d280c52a0 con 0x7f7d280cef30 2024-09-17T03:13:51.011 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.009+0000 7f7d30427640 1 -- 172.21.15.149:0/4195485872 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d280c7c20 con 0x7f7d280cef30 2024-09-17T03:13:51.011 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.010+0000 7f7d1effd640 1 -- 172.21.15.149:0/4195485872 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7d240051c0 con 0x7f7d280cef30 2024-09-17T03:13:51.011 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.010+0000 7f7d1effd640 1 -- 172.21.15.149:0/4195485872 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f7d240176d0 con 0x7f7d280cef30 2024-09-17T03:13:51.011 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.010+0000 7f7d30427640 1 -- 172.21.15.149:0/4195485872 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d2815ffa0 con 0x7f7d280cef30 2024-09-17T03:13:51.012 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.013+0000 7f7d1effd640 1 -- 172.21.15.149:0/4195485872 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f7d24005640 con 0x7f7d280cef30 2024-09-17T03:13:51.082 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.083+0000 7f7d30427640 1 -- 172.21.15.149:0/4195485872 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "osd new", "uuid": "22b83449-856f-4b82-b290-cd0e9e438bbf", "id": 0} v 0) -- 0x7f7d280ceb30 con 0x7f7d280cef30 2024-09-17T03:13:51.086 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.087+0000 7f7d1effd640 1 -- 172.21.15.149:0/4195485872 <== mon.2 v2:172.21.15.149:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "22b83449-856f-4b82-b290-cd0e9e438bbf", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f7d24017a00 con 0x7f7d280cef30 2024-09-17T03:13:51.086 INFO:teuthology.orchestra.run.smithi149.stdout:0 2024-09-17T03:13:51.087 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.088+0000 7f7d30427640 1 -- 172.21.15.149:0/4195485872 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7d280cef30 msgr2=0x7f7d280b1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:51.087 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.088+0000 7f7d30427640 1 --2- 172.21.15.149:0/4195485872 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7d280cef30 0x7f7d280b1d10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7d240099d0 tx=0x7f7d24009e90 comp rx=0 tx=0).stop 2024-09-17T03:13:51.088 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.088+0000 7f7d30427640 1 -- 172.21.15.149:0/4195485872 shutdown_connections 2024-09-17T03:13:51.088 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.088+0000 7f7d30427640 1 --2- 172.21.15.149:0/4195485872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d2815f1d0 0x7f7d28153120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.088 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.088+0000 7f7d30427640 1 --2- 172.21.15.149:0/4195485872 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7d280cef30 0x7f7d280b1d10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.088 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.088+0000 7f7d30427640 1 --2- 172.21.15.149:0/4195485872 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7d2809aa30 0x7f7d280c5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.088 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.088+0000 7f7d30427640 1 -- 172.21.15.149:0/4195485872 >> 172.21.15.149:0/4195485872 conn(0x7f7d280741b0 msgr2=0x7f7d2815c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:51.088 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.088+0000 7f7d30427640 1 -- 172.21.15.149:0/4195485872 shutdown_connections 2024-09-17T03:13:51.088 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.088+0000 7f7d30427640 1 -- 172.21.15.149:0/4195485872 wait complete. 2024-09-17T03:13:51.098 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --cluster ceph osd new 5268ac87-a2b3-4211-bedd-2a869a93511d 1 2024-09-17T03:13:51.213 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.213+0000 7f92ba44d640 1 -- 172.21.15.149:0/614816603 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92b415a100 msgr2=0x7f92b4160100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:51.213 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.213+0000 7f92ba44d640 1 --2- 172.21.15.149:0/614816603 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92b415a100 0x7f92b4160100 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f929c005fd0 tx=0x7f929c02a540 comp rx=0 tx=0).stop 2024-09-17T03:13:51.213 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.213+0000 7f92ba44d640 1 -- 172.21.15.149:0/614816603 shutdown_connections 2024-09-17T03:13:51.213 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.213+0000 7f92ba44d640 1 --2- 172.21.15.149:0/614816603 >> v2:172.21.15.149:3300/0 conn(0x7f92b40b3d20 0x7f92b40b4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.213 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.213+0000 7f92ba44d640 1 --2- 172.21.15.149:0/614816603 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92b415a100 0x7f92b4160100 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.213 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.213+0000 7f92ba44d640 1 -- 172.21.15.149:0/614816603 >> 172.21.15.149:0/614816603 conn(0x7f92b4069330 msgr2=0x7f92b40ced20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:51.213 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.213+0000 7f92ba44d640 1 -- 172.21.15.149:0/614816603 shutdown_connections 2024-09-17T03:13:51.214 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.213+0000 7f92ba44d640 1 -- 172.21.15.149:0/614816603 wait complete. 2024-09-17T03:13:51.214 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.215+0000 7f92ba44d640 1 Processor -- start 2024-09-17T03:13:51.214 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.215+0000 7f92ba44d640 1 -- start start 2024-09-17T03:13:51.217 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.216+0000 7f92ba44d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92b40b3d20 0x7f92b414c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:51.217 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.216+0000 7f92ba44d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92b40b7d70 0x7f92b415ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:51.217 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.216+0000 7f92ba44d640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92b415a100 0x7f92b40ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:51.217 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.216+0000 7f92ba44d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f92b40d0200 con 0x7f92b40b3d20 2024-09-17T03:13:51.217 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.216+0000 7f92ba44d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f92b40d0080 con 0x7f92b40b7d70 2024-09-17T03:13:51.217 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.216+0000 7f92ba44d640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f92b40d0380 con 0x7f92b415a100 2024-09-17T03:13:51.217 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.216+0000 7f92b89ea640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92b415a100 0x7f92b40ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:51.217 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.216+0000 7f92b37fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92b40b7d70 0x7f92b415ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:51.218 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.216+0000 7f92b89ea640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92b415a100 0x7f92b40ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.149:60026/0 (socket says 172.21.15.149:60026) 2024-09-17T03:13:51.218 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.216+0000 7f92b89ea640 1 -- 172.21.15.149:0/3484601946 learned_addr learned my addr 172.21.15.149:0/3484601946 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-09-17T03:13:51.218 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.217+0000 7f92b3fff640 1 --2- 172.21.15.149:0/3484601946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92b40b3d20 0x7f92b414c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:51.218 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.217+0000 7f92b89ea640 1 -- 172.21.15.149:0/3484601946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92b40b7d70 msgr2=0x7f92b415ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:51.218 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.217+0000 7f92b89ea640 1 --2- 172.21.15.149:0/3484601946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92b40b7d70 0x7f92b415ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.218 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.217+0000 7f92b89ea640 1 -- 172.21.15.149:0/3484601946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92b40b3d20 msgr2=0x7f92b414c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:51.218 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.217+0000 7f92b89ea640 1 --2- 172.21.15.149:0/3484601946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92b40b3d20 0x7f92b414c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.218 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.217+0000 7f92b89ea640 1 -- 172.21.15.149:0/3484601946 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92b41f6b10 con 0x7f92b415a100 2024-09-17T03:13:51.218 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.217+0000 7f92b89ea640 1 --2- 172.21.15.149:0/3484601946 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92b415a100 0x7f92b40ca140 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00ec20 tx=0x7f92ac00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:51.218 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.217+0000 7f92b0ff9640 1 -- 172.21.15.149:0/3484601946 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac003b70 con 0x7f92b415a100 2024-09-17T03:13:51.218 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.217+0000 7f92b0ff9640 1 -- 172.21.15.149:0/3484601946 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f92ac003d10 con 0x7f92b415a100 2024-09-17T03:13:51.218 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.217+0000 7f92ba44d640 1 -- 172.21.15.149:0/3484601946 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92b41f6df0 con 0x7f92b415a100 2024-09-17T03:13:51.219 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.217+0000 7f92b0ff9640 1 -- 172.21.15.149:0/3484601946 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ac004280 con 0x7f92b415a100 2024-09-17T03:13:51.219 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.217+0000 7f92ba44d640 1 -- 172.21.15.149:0/3484601946 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f92b41f73b0 con 0x7f92b415a100 2024-09-17T03:13:51.219 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.217+0000 7f92b0ff9640 1 -- 172.21.15.149:0/3484601946 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f92ac021440 con 0x7f92b415a100 2024-09-17T03:13:51.219 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.217+0000 7f92b0ff9640 1 -- 172.21.15.149:0/3484601946 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f92ac0187e0 con 0x7f92b415a100 2024-09-17T03:13:51.220 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.218+0000 7f92ba44d640 1 -- 172.21.15.149:0/3484601946 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92b41f75f0 con 0x7f92b415a100 2024-09-17T03:13:51.221 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.222+0000 7f92b0ff9640 1 -- 172.21.15.149:0/3484601946 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f92b41f75f0 con 0x7f92b415a100 2024-09-17T03:13:51.293 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.294+0000 7f92ba44d640 1 -- 172.21.15.149:0/3484601946 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5268ac87-a2b3-4211-bedd-2a869a93511d", "id": 1} v 0) -- 0x7f92b40b4130 con 0x7f92b415a100 2024-09-17T03:13:51.300 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.300+0000 7f92b0ff9640 1 -- 172.21.15.149:0/3484601946 <== mon.2 v2:172.21.15.149:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5268ac87-a2b3-4211-bedd-2a869a93511d", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f92ac004b40 con 0x7f92b415a100 2024-09-17T03:13:51.300 INFO:teuthology.orchestra.run.smithi149.stdout:1 2024-09-17T03:13:51.301 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.301+0000 7f92ba44d640 1 -- 172.21.15.149:0/3484601946 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92b415a100 msgr2=0x7f92b40ca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:51.301 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.301+0000 7f92ba44d640 1 --2- 172.21.15.149:0/3484601946 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92b415a100 0x7f92b40ca140 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f92ac00ec20 tx=0x7f92ac00c6a0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.301 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.301+0000 7f92ba44d640 1 -- 172.21.15.149:0/3484601946 shutdown_connections 2024-09-17T03:13:51.301 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.301+0000 7f92ba44d640 1 --2- 172.21.15.149:0/3484601946 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f92b415a100 0x7f92b40ca140 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.302 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.301+0000 7f92ba44d640 1 --2- 172.21.15.149:0/3484601946 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f92b40b7d70 0x7f92b415ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.302 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.301+0000 7f92ba44d640 1 --2- 172.21.15.149:0/3484601946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92b40b3d20 0x7f92b414c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.302 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.302+0000 7f92ba44d640 1 -- 172.21.15.149:0/3484601946 >> 172.21.15.149:0/3484601946 conn(0x7f92b4069330 msgr2=0x7f92b415b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:51.302 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.302+0000 7f92ba44d640 1 -- 172.21.15.149:0/3484601946 shutdown_connections 2024-09-17T03:13:51.302 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.302+0000 7f92ba44d640 1 -- 172.21.15.149:0/3484601946 wait complete. 2024-09-17T03:13:51.312 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --cluster ceph osd new a5923363-d620-4f42-8764-b29e6c3095cb 2 2024-09-17T03:13:51.426 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.426+0000 7f9569cf9640 1 -- 172.21.15.149:0/1381951879 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f956415d790 msgr2=0x7f956415dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:51.426 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.426+0000 7f9569cf9640 1 --2- 172.21.15.149:0/1381951879 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f956415d790 0x7f956415dba0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f955c005fd0 tx=0x7f955c02aa90 comp rx=0 tx=0).stop 2024-09-17T03:13:51.427 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.426+0000 7f9569cf9640 1 -- 172.21.15.149:0/1381951879 shutdown_connections 2024-09-17T03:13:51.427 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.426+0000 7f9569cf9640 1 --2- 172.21.15.149:0/1381951879 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f956415d790 0x7f956415dba0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.427 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.426+0000 7f9569cf9640 1 --2- 172.21.15.149:0/1381951879 >> v2:172.21.15.87:3300/0 conn(0x7f956409aa30 0x7f956415c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.427 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.426+0000 7f9569cf9640 1 -- 172.21.15.149:0/1381951879 >> 172.21.15.149:0/1381951879 conn(0x7f95640741b0 msgr2=0x7f95640d1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:51.427 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.426+0000 7f9569cf9640 1 -- 172.21.15.149:0/1381951879 shutdown_connections 2024-09-17T03:13:51.427 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.426+0000 7f9569cf9640 1 -- 172.21.15.149:0/1381951879 wait complete. 2024-09-17T03:13:51.427 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.428+0000 7f9569cf9640 1 Processor -- start 2024-09-17T03:13:51.428 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.428+0000 7f9569cf9640 1 -- start start 2024-09-17T03:13:51.430 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.430+0000 7f9569cf9640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f956409aa30 0x7f95641f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:51.431 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.430+0000 7f9569cf9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95640cef30 0x7f95641f98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:51.431 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.430+0000 7f9569cf9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f956415d790 0x7f95641477d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:51.431 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.430+0000 7f9569cf9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f956415e260 con 0x7f956415d790 2024-09-17T03:13:51.431 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.430+0000 7f9569cf9640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f956415e0e0 con 0x7f95640cef30 2024-09-17T03:13:51.431 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.430+0000 7f9569cf9640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f956415e3e0 con 0x7f956409aa30 2024-09-17T03:13:51.431 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.430+0000 7f95637fe640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f956409aa30 0x7f95641f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:51.431 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.430+0000 7f9562ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95640cef30 0x7f95641f98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:51.431 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.430+0000 7f95637fe640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f956409aa30 0x7f95641f5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.149:60058/0 (socket says 172.21.15.149:60058) 2024-09-17T03:13:51.431 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.430+0000 7f95637fe640 1 -- 172.21.15.149:0/2649508102 learned_addr learned my addr 172.21.15.149:0/2649508102 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-09-17T03:13:51.431 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.430+0000 7f9563fff640 1 --2- 172.21.15.149:0/2649508102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f956415d790 0x7f95641477d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:51.431 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.430+0000 7f95637fe640 1 -- 172.21.15.149:0/2649508102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95640cef30 msgr2=0x7f95641f98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:51.431 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.430+0000 7f95637fe640 1 --2- 172.21.15.149:0/2649508102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95640cef30 0x7f95641f98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.432 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.430+0000 7f95637fe640 1 -- 172.21.15.149:0/2649508102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f956415d790 msgr2=0x7f95641477d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:51.432 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.430+0000 7f95637fe640 1 --2- 172.21.15.149:0/2649508102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f956415d790 0x7f95641477d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.432 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.430+0000 7f95637fe640 1 -- 172.21.15.149:0/2649508102 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95641f60b0 con 0x7f956409aa30 2024-09-17T03:13:51.432 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.430+0000 7f9562ffd640 1 --2- 172.21.15.149:0/2649508102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95640cef30 0x7f95641f98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:13:51.432 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.430+0000 7f9563fff640 1 --2- 172.21.15.149:0/2649508102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f956415d790 0x7f95641477d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:13:51.432 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.430+0000 7f95637fe640 1 --2- 172.21.15.149:0/2649508102 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f956409aa30 0x7f95641f5860 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f955400d770 tx=0x7f955400dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:51.432 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.431+0000 7f954bfff640 1 -- 172.21.15.149:0/2649508102 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554017070 con 0x7f956409aa30 2024-09-17T03:13:51.432 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.431+0000 7f954bfff640 1 -- 172.21.15.149:0/2649508102 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9554009d20 con 0x7f956409aa30 2024-09-17T03:13:51.432 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.431+0000 7f9569cf9640 1 -- 172.21.15.149:0/2649508102 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95641f6390 con 0x7f956409aa30 2024-09-17T03:13:51.432 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.431+0000 7f954bfff640 1 -- 172.21.15.149:0/2649508102 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95540106c0 con 0x7f956409aa30 2024-09-17T03:13:51.432 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.431+0000 7f9569cf9640 1 -- 172.21.15.149:0/2649508102 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9564157c60 con 0x7f956409aa30 2024-09-17T03:13:51.432 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.431+0000 7f954bfff640 1 -- 172.21.15.149:0/2649508102 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f955401a070 con 0x7f956409aa30 2024-09-17T03:13:51.432 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.431+0000 7f954bfff640 1 -- 172.21.15.149:0/2649508102 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f955401cb90 con 0x7f956409aa30 2024-09-17T03:13:51.433 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.432+0000 7f9569cf9640 1 -- 172.21.15.149:0/2649508102 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9564157ec0 con 0x7f956409aa30 2024-09-17T03:13:51.435 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.436+0000 7f954bfff640 1 -- 172.21.15.149:0/2649508102 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f955401c770 con 0x7f956409aa30 2024-09-17T03:13:51.506 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.507+0000 7f9569cf9640 1 -- 172.21.15.149:0/2649508102 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a5923363-d620-4f42-8764-b29e6c3095cb", "id": 2} v 0) -- 0x7f956415e970 con 0x7f956409aa30 2024-09-17T03:13:51.511 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.511+0000 7f954bfff640 1 -- 172.21.15.149:0/2649508102 <== mon.2 v2:172.21.15.149:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a5923363-d620-4f42-8764-b29e6c3095cb", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f955401c950 con 0x7f956409aa30 2024-09-17T03:13:51.511 INFO:teuthology.orchestra.run.smithi149.stdout:2 2024-09-17T03:13:51.512 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.512+0000 7f9569cf9640 1 -- 172.21.15.149:0/2649508102 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f956409aa30 msgr2=0x7f95641f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:51.513 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.512+0000 7f9569cf9640 1 --2- 172.21.15.149:0/2649508102 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f956409aa30 0x7f95641f5860 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f955400d770 tx=0x7f955400dc30 comp rx=0 tx=0).stop 2024-09-17T03:13:51.513 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.512+0000 7f9569cf9640 1 -- 172.21.15.149:0/2649508102 shutdown_connections 2024-09-17T03:13:51.513 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.512+0000 7f9569cf9640 1 --2- 172.21.15.149:0/2649508102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f956415d790 0x7f95641477d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.513 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.512+0000 7f9569cf9640 1 --2- 172.21.15.149:0/2649508102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95640cef30 0x7f95641f98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.513 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.512+0000 7f9569cf9640 1 --2- 172.21.15.149:0/2649508102 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f956409aa30 0x7f95641f5860 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.513 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.512+0000 7f9569cf9640 1 -- 172.21.15.149:0/2649508102 >> 172.21.15.149:0/2649508102 conn(0x7f95640741b0 msgr2=0x7f95640d1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:51.513 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.512+0000 7f9569cf9640 1 -- 172.21.15.149:0/2649508102 shutdown_connections 2024-09-17T03:13:51.513 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.512+0000 7f9569cf9640 1 -- 172.21.15.149:0/2649508102 wait complete. 2024-09-17T03:13:51.523 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --cluster ceph osd new 3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874 3 2024-09-17T03:13:51.638 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.638+0000 7fa4098f9640 1 -- 172.21.15.149:0/1387889108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4040b4430 msgr2=0x7fa404160100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:51.638 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.638+0000 7fa4098f9640 1 --2- 172.21.15.149:0/1387889108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4040b4430 0x7fa404160100 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc005fd0 tx=0x7fa3fc02aaa0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.638 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.638+0000 7fa4098f9640 1 -- 172.21.15.149:0/1387889108 shutdown_connections 2024-09-17T03:13:51.638 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.638+0000 7fa4098f9640 1 --2- 172.21.15.149:0/1387889108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa4040b4430 0x7fa404160100 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.638 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.638+0000 7fa4098f9640 1 -- 172.21.15.149:0/1387889108 >> 172.21.15.149:0/1387889108 conn(0x7fa404069330 msgr2=0x7fa4040ced20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:51.638 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.638+0000 7fa4098f9640 1 -- 172.21.15.149:0/1387889108 shutdown_connections 2024-09-17T03:13:51.638 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.638+0000 7fa4098f9640 1 -- 172.21.15.149:0/1387889108 wait complete. 2024-09-17T03:13:51.639 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.639+0000 7fa4098f9640 1 Processor -- start 2024-09-17T03:13:51.639 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.640+0000 7fa4098f9640 1 -- start start 2024-09-17T03:13:51.642 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.641+0000 7fa4098f9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4040b7d70 0x7fa4040b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:51.642 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.641+0000 7fa4098f9640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa40415a100 0x7fa4040b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:51.642 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.641+0000 7fa4098f9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa40406faa0 0x7fa4040b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:51.642 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.641+0000 7fa4098f9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa4040d03c0 con 0x7fa4040b7d70 2024-09-17T03:13:51.642 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.641+0000 7fa4098f9640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa4040d0240 con 0x7fa40406faa0 2024-09-17T03:13:51.642 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.641+0000 7fa4098f9640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa4040d0540 con 0x7fa40415a100 2024-09-17T03:13:51.642 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.641+0000 7fa4037fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa40406faa0 0x7fa4040b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:51.642 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.641+0000 7fa4027fc640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa40415a100 0x7fa4040b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:51.642 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.641+0000 7fa4037fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa40406faa0 0x7fa4040b2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:57032/0 (socket says 172.21.15.149:57032) 2024-09-17T03:13:51.642 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.641+0000 7fa4037fe640 1 -- 172.21.15.149:0/664987056 learned_addr learned my addr 172.21.15.149:0/664987056 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-09-17T03:13:51.642 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.641+0000 7fa402ffd640 1 --2- 172.21.15.149:0/664987056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4040b7d70 0x7fa4040b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:51.643 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.641+0000 7fa4027fc640 1 -- 172.21.15.149:0/664987056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa40406faa0 msgr2=0x7fa4040b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:51.643 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.641+0000 7fa4027fc640 1 --2- 172.21.15.149:0/664987056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa40406faa0 0x7fa4040b2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.643 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.641+0000 7fa4027fc640 1 -- 172.21.15.149:0/664987056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4040b7d70 msgr2=0x7fa4040b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:51.643 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.641+0000 7fa4027fc640 1 --2- 172.21.15.149:0/664987056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4040b7d70 0x7fa4040b1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.643 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.641+0000 7fa4027fc640 1 -- 172.21.15.149:0/664987056 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa404154ed0 con 0x7fa40415a100 2024-09-17T03:13:51.643 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.641+0000 7fa4037fe640 1 --2- 172.21.15.149:0/664987056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa40406faa0 0x7fa4040b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:13:51.643 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.642+0000 7fa4027fc640 1 --2- 172.21.15.149:0/664987056 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa40415a100 0x7fa4040b20d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00c390 tx=0x7fa3ec00c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:51.643 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.642+0000 7fa3fbfff640 1 -- 172.21.15.149:0/664987056 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec017070 con 0x7fa40415a100 2024-09-17T03:13:51.643 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.642+0000 7fa3fbfff640 1 -- 172.21.15.149:0/664987056 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3ec0053e0 con 0x7fa40415a100 2024-09-17T03:13:51.643 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.642+0000 7fa3fbfff640 1 -- 172.21.15.149:0/664987056 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec003850 con 0x7fa40415a100 2024-09-17T03:13:51.643 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.642+0000 7fa4098f9640 1 -- 172.21.15.149:0/664987056 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4041552f0 con 0x7fa40415a100 2024-09-17T03:13:51.643 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.642+0000 7fa4098f9640 1 -- 172.21.15.149:0/664987056 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4041549b0 con 0x7fa40415a100 2024-09-17T03:13:51.643 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.642+0000 7fa3fbfff640 1 -- 172.21.15.149:0/664987056 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa3ec01a070 con 0x7fa40415a100 2024-09-17T03:13:51.643 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.642+0000 7fa3fbfff640 1 -- 172.21.15.149:0/664987056 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fa3ec011c70 con 0x7fa40415a100 2024-09-17T03:13:51.644 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.643+0000 7fa4098f9640 1 -- 172.21.15.149:0/664987056 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa404160100 con 0x7fa40415a100 2024-09-17T03:13:51.644 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.645+0000 7fa3fbfff640 1 -- 172.21.15.149:0/664987056 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fa3ec021070 con 0x7fa40415a100 2024-09-17T03:13:51.715 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.716+0000 7fa4098f9640 1 -- 172.21.15.149:0/664987056 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874", "id": 3} v 0) -- 0x7fa40415a840 con 0x7fa40415a100 2024-09-17T03:13:51.719 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.720+0000 7fa3fbfff640 1 -- 172.21.15.149:0/664987056 <== mon.2 v2:172.21.15.149:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fa3ec011590 con 0x7fa40415a100 2024-09-17T03:13:51.719 INFO:teuthology.orchestra.run.smithi149.stdout:3 2024-09-17T03:13:51.722 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.721+0000 7fa4098f9640 1 -- 172.21.15.149:0/664987056 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa40415a100 msgr2=0x7fa4040b20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:51.722 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.721+0000 7fa4098f9640 1 --2- 172.21.15.149:0/664987056 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa40415a100 0x7fa4040b20d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00c390 tx=0x7fa3ec00c850 comp rx=0 tx=0).stop 2024-09-17T03:13:51.722 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.721+0000 7fa4098f9640 1 -- 172.21.15.149:0/664987056 shutdown_connections 2024-09-17T03:13:51.722 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.721+0000 7fa4098f9640 1 --2- 172.21.15.149:0/664987056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa40406faa0 0x7fa4040b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.722 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.721+0000 7fa4098f9640 1 --2- 172.21.15.149:0/664987056 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa40415a100 0x7fa4040b20d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.722 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.721+0000 7fa4098f9640 1 --2- 172.21.15.149:0/664987056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa4040b7d70 0x7fa4040b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.722 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.721+0000 7fa4098f9640 1 -- 172.21.15.149:0/664987056 >> 172.21.15.149:0/664987056 conn(0x7fa404069330 msgr2=0x7fa40415acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:51.722 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.721+0000 7fa4098f9640 1 -- 172.21.15.149:0/664987056 shutdown_connections 2024-09-17T03:13:51.722 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.721+0000 7fa4098f9640 1 -- 172.21.15.149:0/664987056 wait complete. 2024-09-17T03:13:51.731 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --cluster ceph osd new 080b1041-3fbe-462c-b735-24f23d52255e 4 2024-09-17T03:13:51.846 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.846+0000 7fc3ff957640 1 -- 172.21.15.149:0/1397032012 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3f815c660 msgr2=0x7fc3f815dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:51.846 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.846+0000 7fc3ff957640 1 --2- 172.21.15.149:0/1397032012 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3f815c660 0x7fc3f815dea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0005fd0 tx=0x7fc3f002a940 comp rx=0 tx=0).stop 2024-09-17T03:13:51.846 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.846+0000 7fc3ff957640 1 -- 172.21.15.149:0/1397032012 shutdown_connections 2024-09-17T03:13:51.846 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.846+0000 7fc3ff957640 1 --2- 172.21.15.149:0/1397032012 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3f815c660 0x7fc3f815dea0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.846 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.846+0000 7fc3ff957640 1 -- 172.21.15.149:0/1397032012 >> 172.21.15.149:0/1397032012 conn(0x7fc3f80741b0 msgr2=0x7fc3f80d1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:51.846 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.846+0000 7fc3ff957640 1 -- 172.21.15.149:0/1397032012 shutdown_connections 2024-09-17T03:13:51.846 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.846+0000 7fc3ff957640 1 -- 172.21.15.149:0/1397032012 wait complete. 2024-09-17T03:13:51.847 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.848+0000 7fc3ff957640 1 Processor -- start 2024-09-17T03:13:51.847 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.848+0000 7fc3ff957640 1 -- start start 2024-09-17T03:13:51.850 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.849+0000 7fc3ff957640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3f809aa30 0x7fc3f81f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:51.850 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.849+0000 7fc3ff957640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc3f815c660 0x7fc3f80ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:51.850 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.849+0000 7fc3ff957640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3f8150760 0x7fc3f80cd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:51.850 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.849+0000 7fc3ff957640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f815e560 con 0x7fc3f8150760 2024-09-17T03:13:51.850 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.849+0000 7fc3ff957640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f815e3e0 con 0x7fc3f809aa30 2024-09-17T03:13:51.851 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.849+0000 7fc3ff957640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc3f815e6e0 con 0x7fc3f815c660 2024-09-17T03:13:51.851 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.849+0000 7fc3fd6f3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3f809aa30 0x7fc3f81f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:51.851 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.849+0000 7fc3fd6f3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3f809aa30 0x7fc3f81f5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:57042/0 (socket says 172.21.15.149:57042) 2024-09-17T03:13:51.851 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.849+0000 7fc3fd6f3640 1 -- 172.21.15.149:0/3792475275 learned_addr learned my addr 172.21.15.149:0/3792475275 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-09-17T03:13:51.851 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.849+0000 7fc3fcef2640 1 --2- 172.21.15.149:0/3792475275 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc3f815c660 0x7fc3f80ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:51.851 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.849+0000 7fc3fdef4640 1 --2- 172.21.15.149:0/3792475275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3f8150760 0x7fc3f80cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:51.851 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.850+0000 7fc3fd6f3640 1 -- 172.21.15.149:0/3792475275 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc3f815c660 msgr2=0x7fc3f80ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:51.851 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.850+0000 7fc3fd6f3640 1 --2- 172.21.15.149:0/3792475275 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc3f815c660 0x7fc3f80ae030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.851 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.850+0000 7fc3fd6f3640 1 -- 172.21.15.149:0/3792475275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3f8150760 msgr2=0x7fc3f80cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:51.851 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.850+0000 7fc3fd6f3640 1 --2- 172.21.15.149:0/3792475275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3f8150760 0x7fc3f80cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.851 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.850+0000 7fc3fd6f3640 1 -- 172.21.15.149:0/3792475275 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f81f6210 con 0x7fc3f809aa30 2024-09-17T03:13:51.851 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.850+0000 7fc3fd6f3640 1 --2- 172.21.15.149:0/3792475275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3f809aa30 0x7fc3f81f5cd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800c490 tx=0x7fc3e800c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:51.851 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.850+0000 7fc3edffb640 1 -- 172.21.15.149:0/3792475275 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8016020 con 0x7fc3f809aa30 2024-09-17T03:13:51.852 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.850+0000 7fc3edffb640 1 -- 172.21.15.149:0/3792475275 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3e8012940 con 0x7fc3f809aa30 2024-09-17T03:13:51.852 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.850+0000 7fc3edffb640 1 -- 172.21.15.149:0/3792475275 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8012d90 con 0x7fc3f809aa30 2024-09-17T03:13:51.852 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.850+0000 7fc3ff957640 1 -- 172.21.15.149:0/3792475275 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f80ce630 con 0x7fc3f809aa30 2024-09-17T03:13:51.852 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.850+0000 7fc3edffb640 1 -- 172.21.15.149:0/3792475275 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc3e8019020 con 0x7fc3f809aa30 2024-09-17T03:13:51.852 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.850+0000 7fc3ff957640 1 -- 172.21.15.149:0/3792475275 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f80ccd50 con 0x7fc3f809aa30 2024-09-17T03:13:51.852 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.851+0000 7fc3edffb640 1 -- 172.21.15.149:0/3792475275 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fc3e801a3b0 con 0x7fc3f809aa30 2024-09-17T03:13:51.853 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.851+0000 7fc3ff957640 1 -- 172.21.15.149:0/3792475275 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f80ceb30 con 0x7fc3f809aa30 2024-09-17T03:13:51.855 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.855+0000 7fc3edffb640 1 -- 172.21.15.149:0/3792475275 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fc3e8020020 con 0x7fc3f809aa30 2024-09-17T03:13:51.924 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.924+0000 7fc3ff957640 1 -- 172.21.15.149:0/3792475275 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd new", "uuid": "080b1041-3fbe-462c-b735-24f23d52255e", "id": 4} v 0) -- 0x7fc3f80cef10 con 0x7fc3f809aa30 2024-09-17T03:13:51.928 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.928+0000 7fc3edffb640 1 -- 172.21.15.149:0/3792475275 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "080b1041-3fbe-462c-b735-24f23d52255e", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fc3f81f6210 con 0x7fc3f809aa30 2024-09-17T03:13:51.928 INFO:teuthology.orchestra.run.smithi149.stdout:4 2024-09-17T03:13:51.929 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.930+0000 7fc3ff957640 1 -- 172.21.15.149:0/3792475275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3f809aa30 msgr2=0x7fc3f81f5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:51.929 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.930+0000 7fc3ff957640 1 --2- 172.21.15.149:0/3792475275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3f809aa30 0x7fc3f81f5cd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc3e800c490 tx=0x7fc3e800c950 comp rx=0 tx=0).stop 2024-09-17T03:13:51.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.930+0000 7fc3ff957640 1 -- 172.21.15.149:0/3792475275 shutdown_connections 2024-09-17T03:13:51.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.930+0000 7fc3ff957640 1 --2- 172.21.15.149:0/3792475275 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3f8150760 0x7fc3f80cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.930+0000 7fc3ff957640 1 --2- 172.21.15.149:0/3792475275 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc3f815c660 0x7fc3f80ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.930+0000 7fc3ff957640 1 --2- 172.21.15.149:0/3792475275 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3f809aa30 0x7fc3f81f5cd0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:51.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.930+0000 7fc3ff957640 1 -- 172.21.15.149:0/3792475275 >> 172.21.15.149:0/3792475275 conn(0x7fc3f80741b0 msgr2=0x7fc3f80b61a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:51.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.930+0000 7fc3ff957640 1 -- 172.21.15.149:0/3792475275 shutdown_connections 2024-09-17T03:13:51.930 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:51.930+0000 7fc3ff957640 1 -- 172.21.15.149:0/3792475275 wait complete. 2024-09-17T03:13:51.940 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --cluster ceph osd new d85dddc7-492e-45f6-a160-ffa21b6e5e2d 5 2024-09-17T03:13:52.055 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.055+0000 7f5e620a4640 1 -- 172.21.15.149:0/3410174969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e5c0744c0 msgr2=0x7f5e5c15f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:52.056 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.055+0000 7f5e620a4640 1 --2- 172.21.15.149:0/3410174969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e5c0744c0 0x7f5e5c15f4b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c1595f0 tx=0x7f5e5402aac0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.056 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.055+0000 7f5e620a4640 1 -- 172.21.15.149:0/3410174969 shutdown_connections 2024-09-17T03:13:52.056 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.055+0000 7f5e620a4640 1 --2- 172.21.15.149:0/3410174969 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e5c0744c0 0x7f5e5c15f4b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.056 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.055+0000 7f5e620a4640 1 --2- 172.21.15.149:0/3410174969 >> v2:172.21.15.87:3300/0 conn(0x7f5e5c15c5e0 0x7f5e5c0cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.056 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.055+0000 7f5e620a4640 1 -- 172.21.15.149:0/3410174969 >> 172.21.15.149:0/3410174969 conn(0x7f5e5c0705d0 msgr2=0x7f5e5c0b2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:52.056 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.056+0000 7f5e620a4640 1 -- 172.21.15.149:0/3410174969 shutdown_connections 2024-09-17T03:13:52.056 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.056+0000 7f5e620a4640 1 -- 172.21.15.149:0/3410174969 wait complete. 2024-09-17T03:13:52.058 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.058+0000 7f5e620a4640 1 Processor -- start 2024-09-17T03:13:52.058 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.059+0000 7f5e620a4640 1 -- start start 2024-09-17T03:13:52.062 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.061+0000 7f5e620a4640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5e5c15c5e0 0x7f5e5c06cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:52.062 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.061+0000 7f5e620a4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e5c15f0a0 0x7f5e5c14f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:52.062 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.061+0000 7f5e620a4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e5c15d860 0x7f5e5c15dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:52.062 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.061+0000 7f5e620a4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c0b5440 con 0x7f5e5c15d860 2024-09-17T03:13:52.062 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.061+0000 7f5e620a4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5c0b52c0 con 0x7f5e5c15f0a0 2024-09-17T03:13:52.062 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.061+0000 7f5e620a4640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5e5c0b55c0 con 0x7f5e5c15c5e0 2024-09-17T03:13:52.062 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.061+0000 7f5e5b7fe640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5e5c15c5e0 0x7f5e5c06cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:52.062 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.061+0000 7f5e5affd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e5c15f0a0 0x7f5e5c14f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:52.063 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.061+0000 7f5e5b7fe640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5e5c15c5e0 0x7f5e5c06cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.149:60084/0 (socket says 172.21.15.149:60084) 2024-09-17T03:13:52.063 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.061+0000 7f5e5b7fe640 1 -- 172.21.15.149:0/2464967361 learned_addr learned my addr 172.21.15.149:0/2464967361 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-09-17T03:13:52.063 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.061+0000 7f5e5bfff640 1 --2- 172.21.15.149:0/2464967361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e5c15d860 0x7f5e5c15dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:52.063 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.061+0000 7f5e5affd640 1 -- 172.21.15.149:0/2464967361 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5e5c15c5e0 msgr2=0x7f5e5c06cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:52.063 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.061+0000 7f5e5affd640 1 --2- 172.21.15.149:0/2464967361 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5e5c15c5e0 0x7f5e5c06cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.063 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.061+0000 7f5e5affd640 1 -- 172.21.15.149:0/2464967361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e5c15d860 msgr2=0x7f5e5c15dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:52.063 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.061+0000 7f5e5affd640 1 --2- 172.21.15.149:0/2464967361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e5c15d860 0x7f5e5c15dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.063 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.061+0000 7f5e5affd640 1 -- 172.21.15.149:0/2464967361 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e5c15e1b0 con 0x7f5e5c15f0a0 2024-09-17T03:13:52.063 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.061+0000 7f5e5b7fe640 1 --2- 172.21.15.149:0/2464967361 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5e5c15c5e0 0x7f5e5c06cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:13:52.063 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.062+0000 7f5e5affd640 1 --2- 172.21.15.149:0/2464967361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e5c15f0a0 0x7f5e5c14f580 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00c3c0 tx=0x7f5e4c00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:52.063 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.062+0000 7f5e53fff640 1 -- 172.21.15.149:0/2464967361 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c017070 con 0x7f5e5c15f0a0 2024-09-17T03:13:52.063 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.062+0000 7f5e53fff640 1 -- 172.21.15.149:0/2464967361 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e4c0053e0 con 0x7f5e5c15f0a0 2024-09-17T03:13:52.063 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.062+0000 7f5e620a4640 1 -- 172.21.15.149:0/2464967361 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e5c1f6b80 con 0x7f5e5c15f0a0 2024-09-17T03:13:52.064 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.062+0000 7f5e53fff640 1 -- 172.21.15.149:0/2464967361 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c003850 con 0x7f5e5c15f0a0 2024-09-17T03:13:52.064 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.062+0000 7f5e620a4640 1 -- 172.21.15.149:0/2464967361 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e5c15b210 con 0x7f5e5c15f0a0 2024-09-17T03:13:52.064 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.062+0000 7f5e53fff640 1 -- 172.21.15.149:0/2464967361 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5e4c01a070 con 0x7f5e5c15f0a0 2024-09-17T03:13:52.064 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.062+0000 7f5e53fff640 1 -- 172.21.15.149:0/2464967361 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f5e4c011d00 con 0x7f5e5c15f0a0 2024-09-17T03:13:52.064 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.063+0000 7f5e620a4640 1 -- 172.21.15.149:0/2464967361 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e5c0b5ae0 con 0x7f5e5c15f0a0 2024-09-17T03:13:52.064 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.065+0000 7f5e53fff640 1 -- 172.21.15.149:0/2464967361 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f5e4c021080 con 0x7f5e5c15f0a0 2024-09-17T03:13:52.135 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.135+0000 7f5e620a4640 1 -- 172.21.15.149:0/2464967361 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d85dddc7-492e-45f6-a160-ffa21b6e5e2d", "id": 5} v 0) -- 0x7f5e5c1515a0 con 0x7f5e5c15f0a0 2024-09-17T03:13:52.138 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.139+0000 7f5e53fff640 1 -- 172.21.15.149:0/2464967361 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d85dddc7-492e-45f6-a160-ffa21b6e5e2d", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f5e5c15e1b0 con 0x7f5e5c15f0a0 2024-09-17T03:13:52.138 INFO:teuthology.orchestra.run.smithi149.stdout:5 2024-09-17T03:13:52.139 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.140+0000 7f5e620a4640 1 -- 172.21.15.149:0/2464967361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e5c15f0a0 msgr2=0x7f5e5c14f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:52.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.140+0000 7f5e620a4640 1 --2- 172.21.15.149:0/2464967361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e5c15f0a0 0x7f5e5c14f580 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c00c3c0 tx=0x7f5e4c00c880 comp rx=0 tx=0).stop 2024-09-17T03:13:52.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.140+0000 7f5e620a4640 1 -- 172.21.15.149:0/2464967361 shutdown_connections 2024-09-17T03:13:52.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.140+0000 7f5e620a4640 1 --2- 172.21.15.149:0/2464967361 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e5c15d860 0x7f5e5c15dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.140+0000 7f5e620a4640 1 --2- 172.21.15.149:0/2464967361 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5e5c15f0a0 0x7f5e5c14f580 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.140+0000 7f5e620a4640 1 --2- 172.21.15.149:0/2464967361 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5e5c15c5e0 0x7f5e5c06cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.140+0000 7f5e620a4640 1 -- 172.21.15.149:0/2464967361 >> 172.21.15.149:0/2464967361 conn(0x7f5e5c0705d0 msgr2=0x7f5e5c071970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:52.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.140+0000 7f5e620a4640 1 -- 172.21.15.149:0/2464967361 shutdown_connections 2024-09-17T03:13:52.140 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.140+0000 7f5e620a4640 1 -- 172.21.15.149:0/2464967361 wait complete. 2024-09-17T03:13:52.150 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --cluster ceph osd new a8a34250-28ce-47f3-a1a4-e298374e4688 6 2024-09-17T03:13:52.266 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.266+0000 7fafb9e4d640 1 -- 172.21.15.149:0/1233971905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb40cf640 msgr2=0x7fafb415c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:52.266 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.266+0000 7fafb9e4d640 1 --2- 172.21.15.149:0/1233971905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb40cf640 0x7fafb415c660 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7faf9c005fd0 tx=0x7faf9c02a670 comp rx=0 tx=0).stop 2024-09-17T03:13:52.266 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.266+0000 7fafb9e4d640 1 -- 172.21.15.149:0/1233971905 shutdown_connections 2024-09-17T03:13:52.266 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.266+0000 7fafb9e4d640 1 --2- 172.21.15.149:0/1233971905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb40cf640 0x7fafb415c660 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.266 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.266+0000 7fafb9e4d640 1 -- 172.21.15.149:0/1233971905 >> 172.21.15.149:0/1233971905 conn(0x7fafb40741b0 msgr2=0x7fafb40d1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:52.266 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.266+0000 7fafb9e4d640 1 -- 172.21.15.149:0/1233971905 shutdown_connections 2024-09-17T03:13:52.266 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.266+0000 7fafb9e4d640 1 -- 172.21.15.149:0/1233971905 wait complete. 2024-09-17T03:13:52.267 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.267+0000 7fafb9e4d640 1 Processor -- start 2024-09-17T03:13:52.267 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.268+0000 7fafb9e4d640 1 -- start start 2024-09-17T03:13:52.270 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.269+0000 7fafb9e4d640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fafb409aa30 0x7fafb40b26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:52.270 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.269+0000 7fafb9e4d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb415d790 0x7fafb41f3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:52.270 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.269+0000 7fafb9e4d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafb41f4480 0x7fafb40b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:52.270 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.269+0000 7fafb9e4d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fafb415e4c0 con 0x7fafb41f4480 2024-09-17T03:13:52.270 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.269+0000 7fafb9e4d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fafb41595f0 con 0x7fafb415d790 2024-09-17T03:13:52.270 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.269+0000 7fafb9e4d640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fafb415e640 con 0x7fafb409aa30 2024-09-17T03:13:52.270 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.269+0000 7fafb2ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb415d790 0x7fafb41f3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:52.270 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.269+0000 7fafb37fe640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fafb409aa30 0x7fafb40b26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:52.270 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.269+0000 7fafb2ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb415d790 0x7fafb41f3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:57084/0 (socket says 172.21.15.149:57084) 2024-09-17T03:13:52.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.269+0000 7fafb2ffd640 1 -- 172.21.15.149:0/121753814 learned_addr learned my addr 172.21.15.149:0/121753814 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-09-17T03:13:52.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.269+0000 7fafb3fff640 1 --2- 172.21.15.149:0/121753814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafb41f4480 0x7fafb40b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:52.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.269+0000 7fafb2ffd640 1 -- 172.21.15.149:0/121753814 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fafb409aa30 msgr2=0x7fafb40b26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:52.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.269+0000 7fafb2ffd640 1 --2- 172.21.15.149:0/121753814 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fafb409aa30 0x7fafb40b26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.269+0000 7fafb2ffd640 1 -- 172.21.15.149:0/121753814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafb41f4480 msgr2=0x7fafb40b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:52.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.269+0000 7fafb2ffd640 1 --2- 172.21.15.149:0/121753814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafb41f4480 0x7fafb40b5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.270+0000 7fafb2ffd640 1 -- 172.21.15.149:0/121753814 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb41f7040 con 0x7fafb415d790 2024-09-17T03:13:52.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.270+0000 7fafb3fff640 1 --2- 172.21.15.149:0/121753814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafb41f4480 0x7fafb40b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:13:52.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.270+0000 7fafb37fe640 1 --2- 172.21.15.149:0/121753814 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fafb409aa30 0x7fafb40b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:13:52.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.270+0000 7fafb2ffd640 1 --2- 172.21.15.149:0/121753814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb415d790 0x7fafb41f3010 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faf9c034040 tx=0x7faf9c0039e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:52.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.270+0000 7fafabfff640 1 -- 172.21.15.149:0/121753814 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c03e030 con 0x7fafb415d790 2024-09-17T03:13:52.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.270+0000 7fafabfff640 1 -- 172.21.15.149:0/121753814 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf9c0047a0 con 0x7fafb415d790 2024-09-17T03:13:52.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.270+0000 7fafb9e4d640 1 -- 172.21.15.149:0/121753814 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafb41f7320 con 0x7fafb415d790 2024-09-17T03:13:52.271 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.270+0000 7fafabfff640 1 -- 172.21.15.149:0/121753814 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c035040 con 0x7fafb415d790 2024-09-17T03:13:52.272 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.270+0000 7fafb9e4d640 1 -- 172.21.15.149:0/121753814 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafb40c5e00 con 0x7fafb415d790 2024-09-17T03:13:52.272 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.270+0000 7fafabfff640 1 -- 172.21.15.149:0/121753814 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faf9c041030 con 0x7fafb415d790 2024-09-17T03:13:52.272 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.270+0000 7fafabfff640 1 -- 172.21.15.149:0/121753814 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7faf9c031400 con 0x7fafb415d790 2024-09-17T03:13:52.272 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.271+0000 7fafb9e4d640 1 -- 172.21.15.149:0/121753814 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafb415c660 con 0x7fafb415d790 2024-09-17T03:13:52.273 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.273+0000 7fafabfff640 1 -- 172.21.15.149:0/121753814 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7faf9c043410 con 0x7fafb415d790 2024-09-17T03:13:52.359 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.359+0000 7fafb9e4d640 1 -- 172.21.15.149:0/121753814 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a8a34250-28ce-47f3-a1a4-e298374e4688", "id": 6} v 0) -- 0x7fafb406dcd0 con 0x7fafb415d790 2024-09-17T03:13:52.365 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.364+0000 7fafabfff640 1 -- 172.21.15.149:0/121753814 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a8a34250-28ce-47f3-a1a4-e298374e4688", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7faf9c0435f0 con 0x7fafb415d790 2024-09-17T03:13:52.365 INFO:teuthology.orchestra.run.smithi149.stdout:6 2024-09-17T03:13:52.366 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.365+0000 7fafb9e4d640 1 -- 172.21.15.149:0/121753814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb415d790 msgr2=0x7fafb41f3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:52.366 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.365+0000 7fafb9e4d640 1 --2- 172.21.15.149:0/121753814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb415d790 0x7fafb41f3010 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faf9c034040 tx=0x7faf9c0039e0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.366 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.365+0000 7fafb9e4d640 1 -- 172.21.15.149:0/121753814 shutdown_connections 2024-09-17T03:13:52.366 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.365+0000 7fafb9e4d640 1 --2- 172.21.15.149:0/121753814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafb41f4480 0x7fafb40b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.366 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.365+0000 7fafb9e4d640 1 --2- 172.21.15.149:0/121753814 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fafb415d790 0x7fafb41f3010 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.366 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.365+0000 7fafb9e4d640 1 --2- 172.21.15.149:0/121753814 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fafb409aa30 0x7fafb40b26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.366 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.365+0000 7fafb9e4d640 1 -- 172.21.15.149:0/121753814 >> 172.21.15.149:0/121753814 conn(0x7fafb40741b0 msgr2=0x7fafb4158890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:52.366 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.365+0000 7fafb9e4d640 1 -- 172.21.15.149:0/121753814 shutdown_connections 2024-09-17T03:13:52.367 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.365+0000 7fafb9e4d640 1 -- 172.21.15.149:0/121753814 wait complete. 2024-09-17T03:13:52.376 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --cluster ceph osd new 33815d16-b7b5-4cb4-80e1-de8012498ccb 7 2024-09-17T03:13:52.488 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.488+0000 7f3448e5c640 1 -- 172.21.15.149:0/1503944203 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f344415c660 msgr2=0x7f344415dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:52.488 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.488+0000 7f3448e5c640 1 --2- 172.21.15.149:0/1503944203 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f344415c660 0x7f344415dea0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f343c009be0 tx=0x7f343c02a920 comp rx=0 tx=0).stop 2024-09-17T03:13:52.489 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.488+0000 7f3448e5c640 1 -- 172.21.15.149:0/1503944203 shutdown_connections 2024-09-17T03:13:52.489 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.488+0000 7f3448e5c640 1 --2- 172.21.15.149:0/1503944203 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f344415c660 0x7f344415dea0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.489 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.488+0000 7f3448e5c640 1 -- 172.21.15.149:0/1503944203 >> 172.21.15.149:0/1503944203 conn(0x7f34440741b0 msgr2=0x7f34440d1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:52.489 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.489+0000 7f3448e5c640 1 -- 172.21.15.149:0/1503944203 shutdown_connections 2024-09-17T03:13:52.489 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.489+0000 7f3448e5c640 1 -- 172.21.15.149:0/1503944203 wait complete. 2024-09-17T03:13:52.490 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.490+0000 7f3448e5c640 1 Processor -- start 2024-09-17T03:13:52.490 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.490+0000 7f3448e5c640 1 -- start start 2024-09-17T03:13:52.493 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.492+0000 7f3448e5c640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f344409aa30 0x7f344414d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:52.493 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.492+0000 7f3448e5c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f344415c660 0x7f34440aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:52.493 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.492+0000 7f3448e5c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34441f8b30 0x7f34440cd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:52.493 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.492+0000 7f3448e5c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f344415e560 con 0x7f34441f8b30 2024-09-17T03:13:52.493 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.492+0000 7f3448e5c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f344415e3e0 con 0x7f344415c660 2024-09-17T03:13:52.493 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.492+0000 7f3448e5c640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f344415e6e0 con 0x7f344409aa30 2024-09-17T03:13:52.494 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.492+0000 7f3441d9b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f344415c660 0x7f34440aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:52.494 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.492+0000 7f344259c640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f344409aa30 0x7f344414d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:52.494 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.492+0000 7f3441d9b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f344415c660 0x7f34440aea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.149:46232/0 (socket says 172.21.15.149:46232) 2024-09-17T03:13:52.494 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.492+0000 7f3441d9b640 1 -- 172.21.15.149:0/581365825 learned_addr learned my addr 172.21.15.149:0/581365825 (peer_addr_for_me v2:172.21.15.149:0/0) 2024-09-17T03:13:52.494 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.492+0000 7f344259c640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f344409aa30 0x7f344414d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.149:38512/0 (socket says 172.21.15.149:38512) 2024-09-17T03:13:52.494 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.492+0000 7f3442d9d640 1 --2- 172.21.15.149:0/581365825 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34441f8b30 0x7f34440cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:52.494 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.492+0000 7f3441d9b640 1 -- 172.21.15.149:0/581365825 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f344409aa30 msgr2=0x7f344414d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:52.494 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.492+0000 7f3441d9b640 1 --2- 172.21.15.149:0/581365825 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f344409aa30 0x7f344414d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.494 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.492+0000 7f3441d9b640 1 -- 172.21.15.149:0/581365825 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34441f8b30 msgr2=0x7f34440cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:52.494 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.492+0000 7f3441d9b640 1 --2- 172.21.15.149:0/581365825 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34441f8b30 0x7f34440cd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.494 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.492+0000 7f3441d9b640 1 -- 172.21.15.149:0/581365825 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34441f92a0 con 0x7f344415c660 2024-09-17T03:13:52.494 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.492+0000 7f344259c640 1 --2- 172.21.15.149:0/581365825 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f344409aa30 0x7f344414d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:13:52.495 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.492+0000 7f3441d9b640 1 --2- 172.21.15.149:0/581365825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f344415c660 0x7f34440aea60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f343400c460 tx=0x7f343400c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:52.495 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.493+0000 7f343affd640 1 -- 172.21.15.149:0/581365825 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3434017070 con 0x7f344415c660 2024-09-17T03:13:52.495 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.493+0000 7f343affd640 1 -- 172.21.15.149:0/581365825 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34340053c0 con 0x7f344415c660 2024-09-17T03:13:52.495 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.493+0000 7f3448e5c640 1 -- 172.21.15.149:0/581365825 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34441f9580 con 0x7f344415c660 2024-09-17T03:13:52.495 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.493+0000 7f343affd640 1 -- 172.21.15.149:0/581365825 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3434003850 con 0x7f344415c660 2024-09-17T03:13:52.495 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.493+0000 7f3448e5c640 1 -- 172.21.15.149:0/581365825 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34440c76d0 con 0x7f344415c660 2024-09-17T03:13:52.495 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.493+0000 7f343affd640 1 -- 172.21.15.149:0/581365825 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f343401a070 con 0x7f344415c660 2024-09-17T03:13:52.495 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.493+0000 7f343affd640 1 -- 172.21.15.149:0/581365825 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f3434011c20 con 0x7f344415c660 2024-09-17T03:13:52.495 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.494+0000 7f3448e5c640 1 -- 172.21.15.149:0/581365825 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34440ceb30 con 0x7f344415c660 2024-09-17T03:13:52.495 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.496+0000 7f343affd640 1 -- 172.21.15.149:0/581365825 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f3434021020 con 0x7f344415c660 2024-09-17T03:13:52.566 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.566+0000 7f3448e5c640 1 -- 172.21.15.149:0/581365825 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd new", "uuid": "33815d16-b7b5-4cb4-80e1-de8012498ccb", "id": 7} v 0) -- 0x7f34440cef10 con 0x7f344415c660 2024-09-17T03:13:52.570 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.570+0000 7f343affd640 1 -- 172.21.15.149:0/581365825 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "33815d16-b7b5-4cb4-80e1-de8012498ccb", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f3434011590 con 0x7f344415c660 2024-09-17T03:13:52.570 INFO:teuthology.orchestra.run.smithi149.stdout:7 2024-09-17T03:13:52.571 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.571+0000 7f3448e5c640 1 -- 172.21.15.149:0/581365825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f344415c660 msgr2=0x7f34440aea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:52.571 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.571+0000 7f3448e5c640 1 --2- 172.21.15.149:0/581365825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f344415c660 0x7f34440aea60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f343400c460 tx=0x7f343400c920 comp rx=0 tx=0).stop 2024-09-17T03:13:52.571 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.571+0000 7f3448e5c640 1 -- 172.21.15.149:0/581365825 shutdown_connections 2024-09-17T03:13:52.571 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.571+0000 7f3448e5c640 1 --2- 172.21.15.149:0/581365825 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34441f8b30 0x7f34440cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.571 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.571+0000 7f3448e5c640 1 --2- 172.21.15.149:0/581365825 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f344415c660 0x7f34440aea60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.572 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.571+0000 7f3448e5c640 1 --2- 172.21.15.149:0/581365825 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f344409aa30 0x7f344414d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.572 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.571+0000 7f3448e5c640 1 -- 172.21.15.149:0/581365825 >> 172.21.15.149:0/581365825 conn(0x7f34440741b0 msgr2=0x7f34440d1360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:52.572 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.571+0000 7f3448e5c640 1 -- 172.21.15.149:0/581365825 shutdown_connections 2024-09-17T03:13:52.572 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:13:52.571+0000 7f3448e5c640 1 -- 172.21.15.149:0/581365825 wait complete. 2024-09-17T03:13:52.582 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-17T03:13:52.582 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-17T03:13:52.584 INFO:tasks.ceph.osd.0:Started 2024-09-17T03:13:52.585 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-17T03:13:52.585 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-17T03:13:52.587 INFO:tasks.ceph.osd.1:Started 2024-09-17T03:13:52.587 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-17T03:13:52.587 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-17T03:13:52.589 INFO:tasks.ceph.osd.2:Started 2024-09-17T03:13:52.589 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-17T03:13:52.589 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-17T03:13:52.591 INFO:tasks.ceph.osd.3:Started 2024-09-17T03:13:52.591 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-17T03:13:52.591 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-17T03:13:52.593 INFO:tasks.ceph.osd.4:Started 2024-09-17T03:13:52.594 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-17T03:13:52.594 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-17T03:13:52.596 INFO:tasks.ceph.osd.5:Started 2024-09-17T03:13:52.596 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-17T03:13:52.596 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-17T03:13:52.598 INFO:tasks.ceph.osd.6:Started 2024-09-17T03:13:52.598 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-17T03:13:52.598 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-17T03:13:52.600 INFO:tasks.ceph.osd.7:Started 2024-09-17T03:13:52.601 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:13:52.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.754+0000 7fbc28207640 1 -- 172.21.15.87:0/46491623 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc2015f890 msgr2=0x7fbc2015bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:52.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.754+0000 7fbc28207640 1 --2- 172.21.15.87:0/46491623 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc2015f890 0x7fbc2015bb90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbc18005fd0 tx=0x7fbc1802a8d0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.754+0000 7fbc28207640 1 -- 172.21.15.87:0/46491623 shutdown_connections 2024-09-17T03:13:52.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.754+0000 7fbc28207640 1 --2- 172.21.15.87:0/46491623 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc2015f890 0x7fbc2015bb90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.754+0000 7fbc28207640 1 --2- 172.21.15.87:0/46491623 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbc20071ed0 0x7fbc200b0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.754+0000 7fbc28207640 1 -- 172.21.15.87:0/46491623 >> 172.21.15.87:0/46491623 conn(0x7fbc200b2d10 msgr2=0x7fbc200b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:52.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.754+0000 7fbc28207640 1 -- 172.21.15.87:0/46491623 shutdown_connections 2024-09-17T03:13:52.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.754+0000 7fbc28207640 1 -- 172.21.15.87:0/46491623 wait complete. 2024-09-17T03:13:52.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.756+0000 7fbc28207640 1 Processor -- start 2024-09-17T03:13:52.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.756+0000 7fbc28207640 1 -- start start 2024-09-17T03:13:52.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.758+0000 7fbc28207640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc20071ed0 0x7fbc201f0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:52.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.758+0000 7fbc28207640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbc20155ab0 0x7fbc200b5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:52.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.758+0000 7fbc28207640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc201f43b0 0x7fbc200b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:52.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.758+0000 7fbc25fa3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc20071ed0 0x7fbc201f0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:52.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.758+0000 7fbc28207640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2015c250 con 0x7fbc20071ed0 2024-09-17T03:13:52.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.758+0000 7fbc28207640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2015c0d0 con 0x7fbc201f43b0 2024-09-17T03:13:52.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.758+0000 7fbc25fa3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc20071ed0 0x7fbc201f0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41130/0 (socket says 172.21.15.87:41130) 2024-09-17T03:13:52.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.758+0000 7fbc28207640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbc2015c3d0 con 0x7fbc20155ab0 2024-09-17T03:13:52.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.758+0000 7fbc25fa3640 1 -- 172.21.15.87:0/424794837 learned_addr learned my addr 172.21.15.87:0/424794837 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:13:52.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.758+0000 7fbc257a2640 1 --2- 172.21.15.87:0/424794837 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbc20155ab0 0x7fbc200b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:52.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.758+0000 7fbc267a4640 1 --2- 172.21.15.87:0/424794837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc201f43b0 0x7fbc200b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:52.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.758+0000 7fbc25fa3640 1 -- 172.21.15.87:0/424794837 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbc20155ab0 msgr2=0x7fbc200b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:52.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.758+0000 7fbc25fa3640 1 --2- 172.21.15.87:0/424794837 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbc20155ab0 0x7fbc200b5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.758+0000 7fbc25fa3640 1 -- 172.21.15.87:0/424794837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc201f43b0 msgr2=0x7fbc200b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:52.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.758+0000 7fbc25fa3640 1 --2- 172.21.15.87:0/424794837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc201f43b0 0x7fbc200b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.758+0000 7fbc25fa3640 1 -- 172.21.15.87:0/424794837 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2015f9f0 con 0x7fbc20071ed0 2024-09-17T03:13:52.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.758+0000 7fbc267a4640 1 --2- 172.21.15.87:0/424794837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc201f43b0 0x7fbc200b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:13:52.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.758+0000 7fbc25fa3640 1 --2- 172.21.15.87:0/424794837 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc20071ed0 0x7fbc201f0660 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbc10009de0 tx=0x7fbc1000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:52.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.759+0000 7fbc167fc640 1 -- 172.21.15.87:0/424794837 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1001c070 con 0x7fbc20071ed0 2024-09-17T03:13:52.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.759+0000 7fbc28207640 1 -- 172.21.15.87:0/424794837 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc201f4b20 con 0x7fbc20071ed0 2024-09-17T03:13:52.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.759+0000 7fbc167fc640 1 -- 172.21.15.87:0/424794837 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc10017400 con 0x7fbc20071ed0 2024-09-17T03:13:52.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.759+0000 7fbc28207640 1 -- 172.21.15.87:0/424794837 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc201f50e0 con 0x7fbc20071ed0 2024-09-17T03:13:52.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.759+0000 7fbc167fc640 1 -- 172.21.15.87:0/424794837 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1001c070 con 0x7fbc20071ed0 2024-09-17T03:13:52.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.759+0000 7fbc167fc640 1 -- 172.21.15.87:0/424794837 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbc1001f070 con 0x7fbc20071ed0 2024-09-17T03:13:52.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.759+0000 7fbc167fc640 1 -- 172.21.15.87:0/424794837 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fbc1000be00 con 0x7fbc20071ed0 2024-09-17T03:13:52.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.760+0000 7fbc28207640 1 -- 172.21.15.87:0/424794837 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc201f52e0 con 0x7fbc20071ed0 2024-09-17T03:13:52.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.762+0000 7fbc167fc640 1 -- 172.21.15.87:0/424794837 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fbc10012070 con 0x7fbc20071ed0 2024-09-17T03:13:52.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.821+0000 7fbc28207640 1 -- 172.21.15.87:0/424794837 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbc2015c8c0 con 0x7fbc20071ed0 2024-09-17T03:13:52.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.821+0000 7fbc167fc640 1 -- 172.21.15.87:0/424794837 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fbc100040e0 con 0x7fbc20071ed0 2024-09-17T03:13:52.820 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:13:52.820 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":10,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:13:52.569440+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:13:52.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.822+0000 7fbc28207640 1 -- 172.21.15.87:0/424794837 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc20071ed0 msgr2=0x7fbc201f0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:52.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.822+0000 7fbc28207640 1 --2- 172.21.15.87:0/424794837 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc20071ed0 0x7fbc201f0660 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbc10009de0 tx=0x7fbc1000f4a0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.822+0000 7fbc28207640 1 -- 172.21.15.87:0/424794837 shutdown_connections 2024-09-17T03:13:52.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.822+0000 7fbc28207640 1 --2- 172.21.15.87:0/424794837 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbc201f43b0 0x7fbc200b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.822+0000 7fbc28207640 1 --2- 172.21.15.87:0/424794837 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbc20155ab0 0x7fbc200b5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.822+0000 7fbc28207640 1 --2- 172.21.15.87:0/424794837 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc20071ed0 0x7fbc201f0660 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:52.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.822+0000 7fbc28207640 1 -- 172.21.15.87:0/424794837 >> 172.21.15.87:0/424794837 conn(0x7fbc200b2d10 msgr2=0x7fbc200d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:52.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.822+0000 7fbc28207640 1 -- 172.21.15.87:0/424794837 shutdown_connections 2024-09-17T03:13:52.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:13:52.822+0000 7fbc28207640 1 -- 172.21.15.87:0/424794837 wait complete. 2024-09-17T03:13:52.832 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-17T03:13:52.832 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-17T03:13:53.639 DEBUG:teuthology.orchestra.run.smithi087:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T03:13:53.745 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.746+0000 7f6fac4d5640 1 -- 172.21.15.87:0/4253593725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f90003a00 msgr2=0x7f6f90003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:53.745 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.746+0000 7f6fac4d5640 1 --2- 172.21.15.87:0/4253593725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6f90003a00 0x7f6f90003e70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000c4a0 tx=0x7f6f90005830 comp rx=0 tx=0).stop 2024-09-17T03:13:53.745 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.746+0000 7f6faaa72640 1 -- 172.21.15.87:0/4253593725 reap_dead start 2024-09-17T03:13:53.745 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.746+0000 7f6fac4d5640 1 -- 172.21.15.87:0/4253593725 shutdown_connections 2024-09-17T03:13:53.745 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.746+0000 7f6fac4d5640 1 -- 172.21.15.87:0/4253593725 >> 172.21.15.87:0/4253593725 conn(0x7f6fa40cdbd0 msgr2=0x7f6fa40cdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:53.745 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.746+0000 7f6fac4d5640 1 -- 172.21.15.87:0/4253593725 shutdown_connections 2024-09-17T03:13:53.746 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.746+0000 7f6fac4d5640 1 -- 172.21.15.87:0/4253593725 wait complete. 2024-09-17T03:13:53.746 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.748+0000 7f6fac4d5640 1 Processor -- start 2024-09-17T03:13:53.746 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.748+0000 7f6fac4d5640 1 -- start start 2024-09-17T03:13:53.749 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.749+0000 7f6fac4d5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f90003a00 0x7f6fa4138c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:53.750 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.749+0000 7f6fac4d5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fa413e0d0 0x7f6fa415dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:53.750 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.749+0000 7f6fac4d5640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6fa415aff0 0x7f6fa415e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:13:53.750 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.749+0000 7f6fac4d5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa4074190 con 0x7f6fa413e0d0 2024-09-17T03:13:53.750 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.749+0000 7f6fac4d5640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6fa4075050 con 0x7f6f90003a00 2024-09-17T03:13:53.750 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.749+0000 7f6fac4d5640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6fa4074310 con 0x7f6fa415aff0 2024-09-17T03:13:53.750 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.749+0000 7f6fa9a70640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fa413e0d0 0x7f6fa415dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:53.750 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.749+0000 7f6fa9a70640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fa413e0d0 0x7f6fa415dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:41140/0 (socket says 172.21.15.87:41140) 2024-09-17T03:13:53.750 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.749+0000 7f6fa9a70640 1 -- 172.21.15.87:0/3766123727 learned_addr learned my addr 172.21.15.87:0/3766123727 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:13:53.750 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.749+0000 7f6faaa72640 1 --2- 172.21.15.87:0/3766123727 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6fa415aff0 0x7f6fa415e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:53.750 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.749+0000 7f6faa271640 1 --2- 172.21.15.87:0/3766123727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f90003a00 0x7f6fa4138c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:13:53.750 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.749+0000 7f6fa9a70640 1 -- 172.21.15.87:0/3766123727 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6fa415aff0 msgr2=0x7f6fa415e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:53.750 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.749+0000 7f6fa9a70640 1 --2- 172.21.15.87:0/3766123727 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6fa415aff0 0x7f6fa415e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:53.750 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.749+0000 7f6fa9a70640 1 -- 172.21.15.87:0/3766123727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f90003a00 msgr2=0x7f6fa4138c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:53.751 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.749+0000 7f6fa9a70640 1 --2- 172.21.15.87:0/3766123727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f90003a00 0x7f6fa4138c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:53.751 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.749+0000 7f6fa9a70640 1 -- 172.21.15.87:0/3766123727 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fa415cb60 con 0x7f6fa413e0d0 2024-09-17T03:13:53.751 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.749+0000 7f6fa9a70640 1 --2- 172.21.15.87:0/3766123727 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fa413e0d0 0x7f6fa415dc00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6f94010a70 tx=0x7f6f94010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:13:53.751 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.750+0000 7f6f9affd640 1 -- 172.21.15.87:0/3766123727 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9400eb90 con 0x7f6fa413e0d0 2024-09-17T03:13:53.751 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.750+0000 7f6f9affd640 1 -- 172.21.15.87:0/3766123727 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f9400ed30 con 0x7f6fa413e0d0 2024-09-17T03:13:53.751 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.750+0000 7f6fac4d5640 1 -- 172.21.15.87:0/3766123727 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fa415ce40 con 0x7f6fa413e0d0 2024-09-17T03:13:53.751 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.750+0000 7f6f9affd640 1 -- 172.21.15.87:0/3766123727 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f94011680 con 0x7f6fa413e0d0 2024-09-17T03:13:53.751 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.750+0000 7f6fac4d5640 1 -- 172.21.15.87:0/3766123727 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fa415d400 con 0x7f6fa413e0d0 2024-09-17T03:13:53.751 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.751+0000 7f6fac4d5640 1 -- 172.21.15.87:0/3766123727 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fa4144eb0 con 0x7f6fa413e0d0 2024-09-17T03:13:53.751 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.751+0000 7f6f9affd640 1 -- 172.21.15.87:0/3766123727 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 2) ==== 50742+0+0 (secure 0 0 0) 0x7f6f94011820 con 0x7f6fa413e0d0 2024-09-17T03:13:53.752 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.751+0000 7f6f9affd640 1 -- 172.21.15.87:0/3766123727 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f6f940531b0 con 0x7f6fa413e0d0 2024-09-17T03:13:53.752 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.753+0000 7f6f9affd640 1 -- 172.21.15.87:0/3766123727 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f6f94018380 con 0x7f6fa413e0d0 2024-09-17T03:13:53.811 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.813+0000 7f6fac4d5640 1 -- 172.21.15.87:0/3766123727 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6fa41441e0 con 0x7f6fa413e0d0 2024-09-17T03:13:53.812 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.813+0000 7f6f9affd640 1 -- 172.21.15.87:0/3766123727 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f6f94021db0 con 0x7f6fa413e0d0 2024-09-17T03:13:53.812 INFO:teuthology.misc.health.smithi087.stdout: 2024-09-17T03:13:53.812 INFO:teuthology.misc.health.smithi087.stdout:{"epoch":10,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:13:52.569440+0000","last_up_change":"0.000000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:13:53.813 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.814+0000 7f6fac4d5640 1 -- 172.21.15.87:0/3766123727 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fa413e0d0 msgr2=0x7f6fa415dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:13:53.813 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.814+0000 7f6fac4d5640 1 --2- 172.21.15.87:0/3766123727 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fa413e0d0 0x7f6fa415dc00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6f94010a70 tx=0x7f6f94010f30 comp rx=0 tx=0).stop 2024-09-17T03:13:53.813 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.814+0000 7f6fac4d5640 1 -- 172.21.15.87:0/3766123727 shutdown_connections 2024-09-17T03:13:53.813 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.814+0000 7f6fac4d5640 1 --2- 172.21.15.87:0/3766123727 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6fa415aff0 0x7f6fa415e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:53.814 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.814+0000 7f6fac4d5640 1 --2- 172.21.15.87:0/3766123727 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fa413e0d0 0x7f6fa415dc00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:53.814 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.814+0000 7f6fac4d5640 1 --2- 172.21.15.87:0/3766123727 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6f90003a00 0x7f6fa4138c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:13:53.814 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.814+0000 7f6fac4d5640 1 -- 172.21.15.87:0/3766123727 >> 172.21.15.87:0/3766123727 conn(0x7f6fa40cdbd0 msgr2=0x7f6fa40b2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:13:53.814 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.814+0000 7f6fac4d5640 1 -- 172.21.15.87:0/3766123727 shutdown_connections 2024-09-17T03:13:53.814 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:13:53.814+0000 7f6fac4d5640 1 -- 172.21.15.87:0/3766123727 wait complete. 2024-09-17T03:13:53.822 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-17T03:13:53.823 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-09-17T03:13:53.822+0000 7fe0c83c5640 -1 Falling back to public interface 2024-09-17T03:13:53.836 INFO:tasks.ceph.osd.2.smithi087.stderr:2024-09-17T03:13:53.837+0000 7fe4a7dbd640 -1 Falling back to public interface 2024-09-17T03:13:54.093 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-09-17T03:13:54.094+0000 7f13fd40f640 -1 Falling back to public interface 2024-09-17T03:13:54.103 INFO:tasks.ceph.osd.1.smithi087.stderr:2024-09-17T03:13:54.104+0000 7f105f21a640 -1 Falling back to public interface 2024-09-17T03:13:54.302 INFO:tasks.ceph.osd.3.smithi087.stderr:2024-09-17T03:13:54.303+0000 7f1b865a4640 -1 Falling back to public interface 2024-09-17T03:13:54.312 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-09-17T03:13:54.313+0000 7fa0eb416640 -1 Falling back to public interface 2024-09-17T03:13:54.346 INFO:tasks.ceph.osd.0.smithi087.stderr:2024-09-17T03:13:54.347+0000 7f8e68bcc640 -1 Falling back to public interface 2024-09-17T03:13:54.420 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-09-17T03:13:54.421+0000 7f47d50c6640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-17T03:13:54.421 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-09-17T03:13:54.421+0000 7f47d50c6640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-17T03:13:54.579 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-09-17T03:13:54.580+0000 7efda981d640 -1 Falling back to public interface 2024-09-17T03:13:57.390 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-09-17T03:13:57.391+0000 7fe0c83c5640 -1 osd.4 0 log_to_monitors true 2024-09-17T03:13:57.620 INFO:tasks.ceph.osd.2.smithi087.stderr:2024-09-17T03:13:57.622+0000 7fe4a7dbd640 -1 osd.2 0 log_to_monitors true 2024-09-17T03:13:57.641 INFO:tasks.ceph.osd.1.smithi087.stderr:2024-09-17T03:13:57.643+0000 7f105f21a640 -1 osd.1 0 log_to_monitors true 2024-09-17T03:13:58.127 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-09-17T03:13:58.128+0000 7f13fd40f640 -1 osd.6 0 log_to_monitors true 2024-09-17T03:13:58.283 INFO:tasks.ceph.osd.3.smithi087.stderr:2024-09-17T03:13:58.284+0000 7f1b865a4640 -1 osd.3 0 log_to_monitors true 2024-09-17T03:13:58.333 INFO:tasks.ceph.osd.0.smithi087.stderr:2024-09-17T03:13:58.334+0000 7f8e68bcc640 -1 osd.0 0 log_to_monitors true 2024-09-17T03:13:58.357 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-09-17T03:13:58.359+0000 7fa0eb416640 -1 osd.7 0 log_to_monitors true 2024-09-17T03:13:58.866 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-09-17T03:13:58.867+0000 7efda981d640 -1 osd.5 0 log_to_monitors true 2024-09-17T03:13:59.370 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-09-17T03:13:59.372+0000 7fe0c4353640 -1 osd.4 0 waiting for initial osdmap 2024-09-17T03:13:59.389 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-09-17T03:13:59.391+0000 7fe0bc149640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T03:14:00.629 DEBUG:teuthology.orchestra.run.smithi087:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T03:14:00.739 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.738+0000 7f89aafed640 1 -- 172.21.15.87:0/4239967568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89a4073460 msgr2=0x7f89a4073870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:00.740 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.738+0000 7f89aafed640 1 --2- 172.21.15.87:0/4239967568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89a4073460 0x7f89a4073870 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f89a40cb510 tx=0x7f899002a560 comp rx=0 tx=0).stop 2024-09-17T03:14:00.740 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.738+0000 7f89aafed640 1 -- 172.21.15.87:0/4239967568 shutdown_connections 2024-09-17T03:14:00.740 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.738+0000 7f89aafed640 1 --2- 172.21.15.87:0/4239967568 >> v2:172.21.15.149:3300/0 conn(0x7f89a406f4c0 0x7f89a406f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:00.740 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.738+0000 7f89aafed640 1 --2- 172.21.15.87:0/4239967568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89a4073460 0x7f89a4073870 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:00.740 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.738+0000 7f89aafed640 1 -- 172.21.15.87:0/4239967568 >> 172.21.15.87:0/4239967568 conn(0x7f89a40b2450 msgr2=0x7f89a40b2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:00.741 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.738+0000 7f89aafed640 1 -- 172.21.15.87:0/4239967568 shutdown_connections 2024-09-17T03:14:00.741 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.738+0000 7f89aafed640 1 -- 172.21.15.87:0/4239967568 wait complete. 2024-09-17T03:14:00.745 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.740+0000 7f89aafed640 1 Processor -- start 2024-09-17T03:14:00.745 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.740+0000 7f89aafed640 1 -- start start 2024-09-17T03:14:00.745 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.741+0000 7f89aafed640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89a406f4c0 0x7f89a4149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:00.745 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.741+0000 7f89aafed640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f89a4073460 0x7f89a41e4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:00.745 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.742+0000 7f89aafed640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a40b5be0 0x7f89a41e5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:00.745 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.742+0000 7f89aafed640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f89a40ce670 con 0x7f89a406f4c0 2024-09-17T03:14:00.745 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.742+0000 7f89aafed640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f89a40ce4f0 con 0x7f89a40b5be0 2024-09-17T03:14:00.745 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.742+0000 7f89aafed640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f89a40ce7f0 con 0x7f89a4073460 2024-09-17T03:14:00.745 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.742+0000 7f89a8d89640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89a406f4c0 0x7f89a4149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:00.745 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.742+0000 7f89a8d89640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89a406f4c0 0x7f89a4149ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35732/0 (socket says 172.21.15.87:35732) 2024-09-17T03:14:00.745 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.742+0000 7f89a8d89640 1 -- 172.21.15.87:0/1995109340 learned_addr learned my addr 172.21.15.87:0/1995109340 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:00.745 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.742+0000 7f89a958a640 1 --2- 172.21.15.87:0/1995109340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a40b5be0 0x7f89a41e5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:00.745 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.742+0000 7f89a3fff640 1 --2- 172.21.15.87:0/1995109340 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f89a4073460 0x7f89a41e4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:00.746 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.742+0000 7f89a8d89640 1 -- 172.21.15.87:0/1995109340 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f89a4073460 msgr2=0x7f89a41e4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:00.746 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.742+0000 7f89a8d89640 1 --2- 172.21.15.87:0/1995109340 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f89a4073460 0x7f89a41e4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:00.746 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.742+0000 7f89a8d89640 1 -- 172.21.15.87:0/1995109340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a40b5be0 msgr2=0x7f89a41e5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:00.746 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.742+0000 7f89a8d89640 1 --2- 172.21.15.87:0/1995109340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a40b5be0 0x7f89a41e5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:00.746 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.742+0000 7f89a8d89640 1 -- 172.21.15.87:0/1995109340 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a415a330 con 0x7f89a406f4c0 2024-09-17T03:14:00.746 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.742+0000 7f89a8d89640 1 --2- 172.21.15.87:0/1995109340 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89a406f4c0 0x7f89a4149ec0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f899800c430 tx=0x7f899800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:00.746 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.742+0000 7f89a17fa640 1 -- 172.21.15.87:0/1995109340 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8998017070 con 0x7f89a406f4c0 2024-09-17T03:14:00.746 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.742+0000 7f89a17fa640 1 -- 172.21.15.87:0/1995109340 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89980053c0 con 0x7f89a406f4c0 2024-09-17T03:14:00.746 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.742+0000 7f89aafed640 1 -- 172.21.15.87:0/1995109340 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a415a610 con 0x7f89a406f4c0 2024-09-17T03:14:00.746 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.742+0000 7f89a17fa640 1 -- 172.21.15.87:0/1995109340 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8998003850 con 0x7f89a406f4c0 2024-09-17T03:14:00.746 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.742+0000 7f89aafed640 1 -- 172.21.15.87:0/1995109340 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a414a800 con 0x7f89a406f4c0 2024-09-17T03:14:00.746 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.744+0000 7f89aafed640 1 -- 172.21.15.87:0/1995109340 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89a40ced40 con 0x7f89a406f4c0 2024-09-17T03:14:00.746 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.744+0000 7f89a17fa640 1 -- 172.21.15.87:0/1995109340 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 4) ==== 101385+0+0 (secure 0 0 0) 0x7f899801a070 con 0x7f89a406f4c0 2024-09-17T03:14:00.747 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.745+0000 7f89a17fa640 1 --2- 172.21.15.87:0/1995109340 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8988078b00 0x7f898807b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:00.747 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.745+0000 7f89a17fa640 1 -- 172.21.15.87:0/1995109340 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2511+0+0 (secure 0 0 0) 0x7f89980992f0 con 0x7f89a406f4c0 2024-09-17T03:14:00.747 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.745+0000 7f89a3fff640 1 --2- 172.21.15.87:0/1995109340 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8988078b00 0x7f898807b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:00.747 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.746+0000 7f89a3fff640 1 --2- 172.21.15.87:0/1995109340 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8988078b00 0x7f898807b010 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8990033000 tx=0x7f8990004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:00.769 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:00.769+0000 7f89a17fa640 1 -- 172.21.15.87:0/1995109340 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f89a40cbf90 con 0x7f89a406f4c0 2024-09-17T03:14:00.791 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-09-17T03:14:00.793+0000 7fa0e6e23640 -1 osd.7 0 waiting for initial osdmap 2024-09-17T03:14:00.807 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-09-17T03:14:00.808+0000 7f13f8d1d640 -1 osd.6 0 waiting for initial osdmap 2024-09-17T03:14:00.825 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-09-17T03:14:00.827+0000 7f13f0b13640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T03:14:00.839 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-09-17T03:14:00.841+0000 7fa0dec19640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T03:14:01.045 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:01.045+0000 7f89aafed640 1 -- 172.21.15.87:0/1995109340 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f89a415a7e0 con 0x7f89a406f4c0 2024-09-17T03:14:01.045 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:01.045+0000 7f89a17fa640 1 -- 172.21.15.87:0/1995109340 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+7590 (secure 0 0 0) 0x7f8998061e40 con 0x7f89a406f4c0 2024-09-17T03:14:01.046 INFO:teuthology.misc.health.smithi087.stdout: 2024-09-17T03:14:01.046 INFO:teuthology.misc.health.smithi087.stdout:{"epoch":14,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:14:00.430724+0000","last_up_change":"2024-09-17T03:13:59.429403+0000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6803","nonce":2414907089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6805","nonce":2414907089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6809","nonce":2414907089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6807","nonce":2414907089}]},"public_addr":"172.21.15.149:6803/2414907089","cluster_addr":"172.21.15.149:6805/2414907089","heartbeat_back_addr":"172.21.15.149:6809/2414907089","heartbeat_front_addr":"172.21.15.149:6807/2414907089","state":["exists","up"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.385791+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:14:01.049 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:01.049+0000 7f89aafed640 1 -- 172.21.15.87:0/1995109340 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8988078b00 msgr2=0x7f898807b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:01.049 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:01.049+0000 7f89aafed640 1 --2- 172.21.15.87:0/1995109340 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8988078b00 0x7f898807b010 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8990033000 tx=0x7f8990004730 comp rx=0 tx=0).stop 2024-09-17T03:14:01.049 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:01.049+0000 7f89aafed640 1 -- 172.21.15.87:0/1995109340 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89a406f4c0 msgr2=0x7f89a4149ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:01.049 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:01.049+0000 7f89aafed640 1 --2- 172.21.15.87:0/1995109340 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89a406f4c0 0x7f89a4149ec0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f899800c430 tx=0x7f899800c8f0 comp rx=0 tx=0).stop 2024-09-17T03:14:01.049 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:01.049+0000 7f89aafed640 1 -- 172.21.15.87:0/1995109340 shutdown_connections 2024-09-17T03:14:01.049 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:01.049+0000 7f89aafed640 1 --2- 172.21.15.87:0/1995109340 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8988078b00 0x7f898807b010 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:01.049 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:01.049+0000 7f89aafed640 1 --2- 172.21.15.87:0/1995109340 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f89a40b5be0 0x7f89a41e5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:01.049 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:01.049+0000 7f89aafed640 1 --2- 172.21.15.87:0/1995109340 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f89a4073460 0x7f89a41e4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:01.050 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:01.049+0000 7f89aafed640 1 --2- 172.21.15.87:0/1995109340 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89a406f4c0 0x7f89a4149ec0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:01.050 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:01.049+0000 7f89aafed640 1 -- 172.21.15.87:0/1995109340 >> 172.21.15.87:0/1995109340 conn(0x7f89a40b2450 msgr2=0x7f89a40b2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:01.050 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:01.049+0000 7f89aafed640 1 -- 172.21.15.87:0/1995109340 shutdown_connections 2024-09-17T03:14:01.050 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:01.049+0000 7f89aafed640 1 -- 172.21.15.87:0/1995109340 wait complete. 2024-09-17T03:14:01.064 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-17T03:14:01.423 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-09-17T03:14:01.425+0000 7efda51e7640 -1 osd.5 0 waiting for initial osdmap 2024-09-17T03:14:01.434 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-09-17T03:14:01.436+0000 7efd9cfdd640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T03:14:01.555 INFO:tasks.ceph.osd.3.smithi087.stderr:2024-09-17T03:14:01.555+0000 7f1b82532640 -1 osd.3 0 waiting for initial osdmap 2024-09-17T03:14:01.557 INFO:tasks.ceph.osd.2.smithi087.stderr:2024-09-17T03:14:01.558+0000 7fe4a3d4b640 -1 osd.2 0 waiting for initial osdmap 2024-09-17T03:14:01.567 INFO:tasks.ceph.osd.3.smithi087.stderr:2024-09-17T03:14:01.568+0000 7f1b7a328640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T03:14:01.576 INFO:tasks.ceph.osd.2.smithi087.stderr:2024-09-17T03:14:01.576+0000 7fe49bb41640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T03:14:01.625 INFO:tasks.ceph.osd.0.smithi087.stderr:2024-09-17T03:14:01.625+0000 7f8e6536f640 -1 osd.0 0 waiting for initial osdmap 2024-09-17T03:14:01.635 INFO:tasks.ceph.osd.0.smithi087.stderr:2024-09-17T03:14:01.636+0000 7f8e5c950640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T03:14:01.636 INFO:tasks.ceph.osd.1.smithi087.stderr:2024-09-17T03:14:01.637+0000 7f105abe7640 -1 osd.1 0 waiting for initial osdmap 2024-09-17T03:14:01.647 INFO:tasks.ceph.osd.1.smithi087.stderr:2024-09-17T03:14:01.648+0000 7f10529dd640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-17T03:14:07.871 DEBUG:teuthology.orchestra.run.smithi087:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-17T03:14:07.987 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.987+0000 7f956f486640 1 -- 172.21.15.87:0/1737790436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95680b7de0 msgr2=0x7f9568160c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:07.987 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.987+0000 7f956f486640 1 --2- 172.21.15.87:0/1737790436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95680b7de0 0x7f9568160c80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f95680b4820 tx=0x7f956402a420 comp rx=0 tx=0).stop 2024-09-17T03:14:07.987 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.987+0000 7f956f486640 1 -- 172.21.15.87:0/1737790436 shutdown_connections 2024-09-17T03:14:07.987 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.987+0000 7f956f486640 1 --2- 172.21.15.87:0/1737790436 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95680b4260 0x7f95680710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:07.987 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.987+0000 7f956f486640 1 --2- 172.21.15.87:0/1737790436 >> v2:172.21.15.149:3300/0 conn(0x7f956815c620 0x7f95680b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:07.988 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.987+0000 7f956f486640 1 --2- 172.21.15.87:0/1737790436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95680b7de0 0x7f9568160c80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:07.988 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.987+0000 7f956f486640 1 -- 172.21.15.87:0/1737790436 >> 172.21.15.87:0/1737790436 conn(0x7f9568069450 msgr2=0x7f95680ced20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:07.988 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.987+0000 7f956f486640 1 -- 172.21.15.87:0/1737790436 shutdown_connections 2024-09-17T03:14:07.988 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.987+0000 7f956f486640 1 -- 172.21.15.87:0/1737790436 wait complete. 2024-09-17T03:14:07.988 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.989+0000 7f956f486640 1 Processor -- start 2024-09-17T03:14:07.989 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.989+0000 7f956f486640 1 -- start start 2024-09-17T03:14:07.991 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.990+0000 7f956f486640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95680b4260 0x7f956815db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:07.991 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.990+0000 7f956f486640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95680b7de0 0x7f95681f69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:07.992 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.990+0000 7f956f486640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f956815c620 0x7f95680c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:07.992 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.990+0000 7f956f486640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f95680719b0 con 0x7f956815c620 2024-09-17T03:14:07.992 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.990+0000 7f956f486640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9568071830 con 0x7f95680b4260 2024-09-17T03:14:07.992 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.990+0000 7f956f486640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9568071b30 con 0x7f95680b7de0 2024-09-17T03:14:07.992 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.990+0000 7f956da23640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f956815c620 0x7f95680c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:07.992 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.990+0000 7f956da23640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f956815c620 0x7f95680c81e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50128/0 (socket says 172.21.15.87:50128) 2024-09-17T03:14:07.992 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.990+0000 7f956da23640 1 -- 172.21.15.87:0/1552099731 learned_addr learned my addr 172.21.15.87:0/1552099731 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:07.992 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.991+0000 7f956d222640 1 --2- 172.21.15.87:0/1552099731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95680b4260 0x7f956815db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:07.992 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.991+0000 7f956ca21640 1 --2- 172.21.15.87:0/1552099731 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95680b7de0 0x7f95681f69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:07.992 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.991+0000 7f956da23640 1 -- 172.21.15.87:0/1552099731 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95680b7de0 msgr2=0x7f95681f69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:07.992 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.991+0000 7f956da23640 1 --2- 172.21.15.87:0/1552099731 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95680b7de0 0x7f95681f69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:07.992 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.991+0000 7f956da23640 1 -- 172.21.15.87:0/1552099731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95680b4260 msgr2=0x7f956815db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:07.993 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.991+0000 7f956da23640 1 --2- 172.21.15.87:0/1552099731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95680b4260 0x7f956815db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:07.993 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.991+0000 7f956da23640 1 -- 172.21.15.87:0/1552099731 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956815e570 con 0x7f956815c620 2024-09-17T03:14:07.993 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.991+0000 7f956d222640 1 --2- 172.21.15.87:0/1552099731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95680b4260 0x7f956815db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:14:07.993 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.991+0000 7f956da23640 1 --2- 172.21.15.87:0/1552099731 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f956815c620 0x7f95680c81e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f956000eab0 tx=0x7f9560009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:07.993 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.991+0000 7f955dffb640 1 -- 172.21.15.87:0/1552099731 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f956000cde0 con 0x7f956815c620 2024-09-17T03:14:07.993 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.991+0000 7f955dffb640 1 -- 172.21.15.87:0/1552099731 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9560004c10 con 0x7f956815c620 2024-09-17T03:14:07.993 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.991+0000 7f956f486640 1 -- 172.21.15.87:0/1552099731 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f956815e850 con 0x7f956815c620 2024-09-17T03:14:07.993 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.991+0000 7f955dffb640 1 -- 172.21.15.87:0/1552099731 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9560005020 con 0x7f956815c620 2024-09-17T03:14:07.993 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.991+0000 7f956f486640 1 -- 172.21.15.87:0/1552099731 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f956815ee10 con 0x7f956815c620 2024-09-17T03:14:07.994 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.992+0000 7f956f486640 1 -- 172.21.15.87:0/1552099731 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95680b75c0 con 0x7f956815c620 2024-09-17T03:14:07.994 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.993+0000 7f955dffb640 1 -- 172.21.15.87:0/1552099731 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f95600051c0 con 0x7f956815c620 2024-09-17T03:14:07.994 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.993+0000 7f955dffb640 1 --2- 172.21.15.87:0/1552099731 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f954c078b90 0x7f954c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:07.994 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.993+0000 7f955dffb640 1 -- 172.21.15.87:0/1552099731 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f956009b7f0 con 0x7f956815c620 2024-09-17T03:14:07.994 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.994+0000 7f956d222640 1 --2- 172.21.15.87:0/1552099731 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f954c078b90 0x7f954c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:07.994 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.994+0000 7f956d222640 1 --2- 172.21.15.87:0/1552099731 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f954c078b90 0x7f954c07b0a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9564033000 tx=0x7f9564004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:07.998 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:07.998+0000 7f955dffb640 1 -- 172.21.15.87:0/1552099731 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9560063f80 con 0x7f956815c620 2024-09-17T03:14:08.174 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:08.174+0000 7f956f486640 1 -- 172.21.15.87:0/1552099731 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9568160c30 con 0x7f956815c620 2024-09-17T03:14:08.175 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:08.175+0000 7f955dffb640 1 -- 172.21.15.87:0/1552099731 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12920 (secure 0 0 0) 0x7f9560063880 con 0x7f956815c620 2024-09-17T03:14:08.175 INFO:teuthology.misc.health.smithi087.stdout: 2024-09-17T03:14:08.175 INFO:teuthology.misc.health.smithi087.stdout:{"epoch":19,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:14:05.859889+0000","last_up_change":"2024-09-17T03:14:01.833605+0000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:14:03.433404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6825","nonce":2430409749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6827","nonce":2430409749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6831","nonce":2430409749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6829","nonce":2430409749}]},"public_addr":"172.21.15.87:6825/2430409749","cluster_addr":"172.21.15.87:6827/2430409749","heartbeat_back_addr":"172.21.15.87:6831/2430409749","heartbeat_front_addr":"172.21.15.87:6829/2430409749","state":["exists","up"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6809","nonce":3467605435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6811","nonce":3467605435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6815","nonce":3467605435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6813","nonce":3467605435}]},"public_addr":"172.21.15.87:6809/3467605435","cluster_addr":"172.21.15.87:6811/3467605435","heartbeat_back_addr":"172.21.15.87:6815/3467605435","heartbeat_front_addr":"172.21.15.87:6813/3467605435","state":["exists","up"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6801","nonce":1450834766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6803","nonce":1450834766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6807","nonce":1450834766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6805","nonce":1450834766}]},"public_addr":"172.21.15.87:6801/1450834766","cluster_addr":"172.21.15.87:6803/1450834766","heartbeat_back_addr":"172.21.15.87:6807/1450834766","heartbeat_front_addr":"172.21.15.87:6805/1450834766","state":["exists","up"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6817","nonce":4253161216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6819","nonce":4253161216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6823","nonce":4253161216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6821","nonce":4253161216}]},"public_addr":"172.21.15.87:6817/4253161216","cluster_addr":"172.21.15.87:6819/4253161216","heartbeat_back_addr":"172.21.15.87:6823/4253161216","heartbeat_front_addr":"172.21.15.87:6821/4253161216","state":["exists","up"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6803","nonce":2414907089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6805","nonce":2414907089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6809","nonce":2414907089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6807","nonce":2414907089}]},"public_addr":"172.21.15.149:6803/2414907089","cluster_addr":"172.21.15.149:6805/2414907089","heartbeat_back_addr":"172.21.15.149:6809/2414907089","heartbeat_front_addr":"172.21.15.149:6807/2414907089","state":["exists","up"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6827","nonce":2182999530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6829","nonce":2182999530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6833","nonce":2182999530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6831","nonce":2182999530}]},"public_addr":"172.21.15.149:6827/2182999530","cluster_addr":"172.21.15.149:6829/2182999530","heartbeat_back_addr":"172.21.15.149:6833/2182999530","heartbeat_front_addr":"172.21.15.149:6831/2182999530","state":["exists","up"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6811","nonce":4289519146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6813","nonce":4289519146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6817","nonce":4289519146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6815","nonce":4289519146}]},"public_addr":"172.21.15.149:6811/4289519146","cluster_addr":"172.21.15.149:6813/4289519146","heartbeat_back_addr":"172.21.15.149:6817/4289519146","heartbeat_front_addr":"172.21.15.149:6815/4289519146","state":["exists","up"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6819","nonce":2771010477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6821","nonce":2771010477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6825","nonce":2771010477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6823","nonce":2771010477}]},"public_addr":"172.21.15.149:6819/2771010477","cluster_addr":"172.21.15.149:6821/2771010477","heartbeat_back_addr":"172.21.15.149:6825/2771010477","heartbeat_front_addr":"172.21.15.149:6823/2771010477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.313297+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.642861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.654781+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.302606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.385791+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.915266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.121365+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.394432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:14:08.177 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:08.177+0000 7f956f486640 1 -- 172.21.15.87:0/1552099731 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f954c078b90 msgr2=0x7f954c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:08.177 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:08.177+0000 7f956f486640 1 --2- 172.21.15.87:0/1552099731 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f954c078b90 0x7f954c07b0a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9564033000 tx=0x7f9564004730 comp rx=0 tx=0).stop 2024-09-17T03:14:08.178 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:08.177+0000 7f956f486640 1 -- 172.21.15.87:0/1552099731 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f956815c620 msgr2=0x7f95680c81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:08.178 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:08.177+0000 7f956f486640 1 --2- 172.21.15.87:0/1552099731 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f956815c620 0x7f95680c81e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f956000eab0 tx=0x7f9560009f70 comp rx=0 tx=0).stop 2024-09-17T03:14:08.178 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:08.177+0000 7f956f486640 1 -- 172.21.15.87:0/1552099731 shutdown_connections 2024-09-17T03:14:08.178 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:08.177+0000 7f956f486640 1 --2- 172.21.15.87:0/1552099731 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f954c078b90 0x7f954c07b0a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:08.178 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:08.177+0000 7f956f486640 1 --2- 172.21.15.87:0/1552099731 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f956815c620 0x7f95680c81e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:08.178 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:08.177+0000 7f956f486640 1 --2- 172.21.15.87:0/1552099731 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95680b7de0 0x7f95681f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:08.178 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:08.177+0000 7f956f486640 1 --2- 172.21.15.87:0/1552099731 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95680b4260 0x7f956815db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:08.178 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:08.177+0000 7f956f486640 1 -- 172.21.15.87:0/1552099731 >> 172.21.15.87:0/1552099731 conn(0x7f9568069450 msgr2=0x7f95680ced20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:08.178 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:08.177+0000 7f956f486640 1 -- 172.21.15.87:0/1552099731 shutdown_connections 2024-09-17T03:14:08.178 INFO:teuthology.misc.health.smithi087.stderr:2024-09-17T03:14:08.177+0000 7f956f486640 1 -- 172.21.15.87:0/1552099731 wait complete. 2024-09-17T03:14:08.187 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-17T03:14:08.187 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-17T03:14:08.187 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T03:14:08.188 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T03:14:08.190 INFO:tasks.ceph.mds.a:Started 2024-09-17T03:14:08.190 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T03:14:08.190 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T03:14:08.192 INFO:tasks.ceph.mds.c:Started 2024-09-17T03:14:08.192 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T03:14:08.192 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T03:14:08.195 INFO:tasks.ceph.mds.b:Started 2024-09-17T03:14:08.195 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T03:14:08.195 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T03:14:08.197 INFO:tasks.ceph.mds.d:Started 2024-09-17T03:14:08.197 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-17T03:14:08.198 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:14:08.274 INFO:tasks.ceph.mds.c.smithi087.stdout:starting mds.c at 2024-09-17T03:14:08.274 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-09-17T03:14:08.274 INFO:tasks.ceph.mds.a.smithi087.stderr:2024-09-17T03:14:08.275+0000 7f5cee049640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6833/0: (98) Address already in use 2024-09-17T03:14:08.298 INFO:tasks.ceph.mds.b.smithi149.stdout:starting mds.b at 2024-09-17T03:14:08.299 INFO:tasks.ceph.mds.d.smithi149.stdout:starting mds.d at 2024-09-17T03:14:08.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.317+0000 7f0969d4a640 1 -- 172.21.15.87:0/317467531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09640ccf80 msgr2=0x7f09640cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:08.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.317+0000 7f0969d4a640 1 --2- 172.21.15.87:0/317467531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09640ccf80 0x7f09640cd390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f095c005fd0 tx=0x7f095c02a8b0 comp rx=0 tx=0).stop 2024-09-17T03:14:08.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.317+0000 7f0969d4a640 1 -- 172.21.15.87:0/317467531 shutdown_connections 2024-09-17T03:14:08.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.317+0000 7f0969d4a640 1 --2- 172.21.15.87:0/317467531 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09640ccf80 0x7f09640cd390 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:08.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.317+0000 7f0969d4a640 1 --2- 172.21.15.87:0/317467531 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09640d0b40 0x7f09640d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:08.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.317+0000 7f0969d4a640 1 -- 172.21.15.87:0/317467531 >> 172.21.15.87:0/317467531 conn(0x7f09640d2230 msgr2=0x7f09640d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:08.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.317+0000 7f0969d4a640 1 -- 172.21.15.87:0/317467531 shutdown_connections 2024-09-17T03:14:08.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.317+0000 7f0969d4a640 1 -- 172.21.15.87:0/317467531 wait complete. 2024-09-17T03:14:08.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.319+0000 7f0969d4a640 1 Processor -- start 2024-09-17T03:14:08.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.319+0000 7f0969d4a640 1 -- start start 2024-09-17T03:14:08.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.321+0000 7f0969d4a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09640b76d0 0x7f09641e1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:08.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.321+0000 7f0969d4a640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09640ccf80 0x7f09641e8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:08.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.321+0000 7f0969d4a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09640d0b40 0x7f09641cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:08.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.321+0000 7f0969d4a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f09640b3040 con 0x7f09640b76d0 2024-09-17T03:14:08.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.321+0000 7f0969d4a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f09640b2ec0 con 0x7f09640d0b40 2024-09-17T03:14:08.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.321+0000 7f0969d4a640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f09640b19d0 con 0x7f09640ccf80 2024-09-17T03:14:08.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.321+0000 7f09637fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09640b76d0 0x7f09641e1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:08.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.321+0000 7f09637fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09640b76d0 0x7f09641e1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50164/0 (socket says 172.21.15.87:50164) 2024-09-17T03:14:08.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.321+0000 7f09637fe640 1 -- 172.21.15.87:0/1848189887 learned_addr learned my addr 172.21.15.87:0/1848189887 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:08.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.321+0000 7f0963fff640 1 --2- 172.21.15.87:0/1848189887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09640d0b40 0x7f09641cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:08.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.321+0000 7f0962ffd640 1 --2- 172.21.15.87:0/1848189887 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09640ccf80 0x7f09641e8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:08.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.321+0000 7f09637fe640 1 -- 172.21.15.87:0/1848189887 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09640ccf80 msgr2=0x7f09641e8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:08.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.321+0000 7f09637fe640 1 --2- 172.21.15.87:0/1848189887 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09640ccf80 0x7f09641e8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:08.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.321+0000 7f09637fe640 1 -- 172.21.15.87:0/1848189887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09640d0b40 msgr2=0x7f09641cce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:08.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.321+0000 7f09637fe640 1 --2- 172.21.15.87:0/1848189887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09640d0b40 0x7f09641cce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:08.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.321+0000 7f09637fe640 1 -- 172.21.15.87:0/1848189887 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09641e02f0 con 0x7f09640b76d0 2024-09-17T03:14:08.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.321+0000 7f09637fe640 1 --2- 172.21.15.87:0/1848189887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09640b76d0 0x7f09641e1390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f095400c3e0 tx=0x7f095400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:08.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.321+0000 7f094bfff640 1 -- 172.21.15.87:0/1848189887 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954011790 con 0x7f09640b76d0 2024-09-17T03:14:08.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.321+0000 7f094bfff640 1 -- 172.21.15.87:0/1848189887 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0954017070 con 0x7f09640b76d0 2024-09-17T03:14:08.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.321+0000 7f0969d4a640 1 -- 172.21.15.87:0/1848189887 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09641e05d0 con 0x7f09640b76d0 2024-09-17T03:14:08.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.321+0000 7f094bfff640 1 -- 172.21.15.87:0/1848189887 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954011ce0 con 0x7f09640b76d0 2024-09-17T03:14:08.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.321+0000 7f0969d4a640 1 -- 172.21.15.87:0/1848189887 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09641e0b90 con 0x7f09640b76d0 2024-09-17T03:14:08.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.322+0000 7f0969d4a640 1 -- 172.21.15.87:0/1848189887 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09640d0f50 con 0x7f09640b76d0 2024-09-17T03:14:08.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.323+0000 7f094bfff640 1 -- 172.21.15.87:0/1848189887 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f095401c460 con 0x7f09640b76d0 2024-09-17T03:14:08.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.324+0000 7f094bfff640 1 --2- 172.21.15.87:0/1848189887 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0950078b90 0x7f095007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:08.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.324+0000 7f094bfff640 1 -- 172.21.15.87:0/1848189887 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f095409a7f0 con 0x7f09640b76d0 2024-09-17T03:14:08.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.324+0000 7f0962ffd640 1 --2- 172.21.15.87:0/1848189887 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0950078b90 0x7f095007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:08.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.324+0000 7f0962ffd640 1 --2- 172.21.15.87:0/1848189887 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0950078b90 0x7f095007b0a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f09641dfde0 tx=0x7f0958005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:08.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.328+0000 7f094bfff640 1 -- 172.21.15.87:0/1848189887 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0954063040 con 0x7f09640b76d0 2024-09-17T03:14:08.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.539+0000 7f0969d4a640 1 -- 172.21.15.87:0/1848189887 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09640b0e90 con 0x7f09640b76d0 2024-09-17T03:14:08.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.539+0000 7f094bfff640 1 -- 172.21.15.87:0/1848189887 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1431 (secure 0 0 0) 0x7f0954062940 con 0x7f09640b76d0 2024-09-17T03:14:08.539 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:08.540 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":2,"btime":"2024-09-17T03:14:08:280269+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/666860906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":666860906},{"type":"v1","addr":"172.21.15.87:6833","nonce":666860906}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-17T03:14:08.540 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 2 2024-09-17T03:14:08.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.541+0000 7f0969d4a640 1 -- 172.21.15.87:0/1848189887 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0950078b90 msgr2=0x7f095007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:08.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.541+0000 7f0969d4a640 1 --2- 172.21.15.87:0/1848189887 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0950078b90 0x7f095007b0a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f09641dfde0 tx=0x7f0958005f90 comp rx=0 tx=0).stop 2024-09-17T03:14:08.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.541+0000 7f0969d4a640 1 -- 172.21.15.87:0/1848189887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09640b76d0 msgr2=0x7f09641e1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:08.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.541+0000 7f0969d4a640 1 --2- 172.21.15.87:0/1848189887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09640b76d0 0x7f09641e1390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f095400c3e0 tx=0x7f095400c8a0 comp rx=0 tx=0).stop 2024-09-17T03:14:08.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.542+0000 7f0969d4a640 1 -- 172.21.15.87:0/1848189887 shutdown_connections 2024-09-17T03:14:08.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.542+0000 7f0969d4a640 1 --2- 172.21.15.87:0/1848189887 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0950078b90 0x7f095007b0a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:08.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.542+0000 7f0969d4a640 1 --2- 172.21.15.87:0/1848189887 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f09640d0b40 0x7f09641cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:08.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.542+0000 7f0969d4a640 1 --2- 172.21.15.87:0/1848189887 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f09640ccf80 0x7f09641e8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:08.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.542+0000 7f0969d4a640 1 --2- 172.21.15.87:0/1848189887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09640b76d0 0x7f09641e1390 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:08.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.542+0000 7f0969d4a640 1 -- 172.21.15.87:0/1848189887 >> 172.21.15.87:0/1848189887 conn(0x7f09640d2230 msgr2=0x7f09641e4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:08.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.542+0000 7f0969d4a640 1 -- 172.21.15.87:0/1848189887 shutdown_connections 2024-09-17T03:14:08.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:08.542+0000 7f0969d4a640 1 -- 172.21.15.87:0/1848189887 wait complete. 2024-09-17T03:14:08.841 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:14:08.843 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:14:10.553 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:14:10.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.676+0000 7fd52cca4640 1 -- 172.21.15.87:0/2626887467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5280cf020 msgr2=0x7fd5280b5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:10.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.676+0000 7fd52cca4640 1 --2- 172.21.15.87:0/2626887467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5280cf020 0x7fd5280b5ea0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd51c005fd0 tx=0x7fd51c02a670 comp rx=0 tx=0).stop 2024-09-17T03:14:10.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.676+0000 7fd52cca4640 1 -- 172.21.15.87:0/2626887467 shutdown_connections 2024-09-17T03:14:10.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.676+0000 7fd52cca4640 1 --2- 172.21.15.87:0/2626887467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5280cf020 0x7fd5280b5ea0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:10.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.676+0000 7fd52cca4640 1 -- 172.21.15.87:0/2626887467 >> 172.21.15.87:0/2626887467 conn(0x7fd528069900 msgr2=0x7fd5280d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:10.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.676+0000 7fd52cca4640 1 -- 172.21.15.87:0/2626887467 shutdown_connections 2024-09-17T03:14:10.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.676+0000 7fd52cca4640 1 -- 172.21.15.87:0/2626887467 wait complete. 2024-09-17T03:14:10.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.678+0000 7fd52cca4640 1 Processor -- start 2024-09-17T03:14:10.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.678+0000 7fd52cca4640 1 -- start start 2024-09-17T03:14:10.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.679+0000 7fd52cca4640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5280cf020 0x7fd5280cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:10.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.679+0000 7fd52cca4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5280cf430 0x7fd5281f9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:10.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.679+0000 7fd52cca4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd52815c5f0 0x7fd528149100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:10.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.679+0000 7fd52cca4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd5280b67b0 con 0x7fd52815c5f0 2024-09-17T03:14:10.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.679+0000 7fd52cca4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd5280b6630 con 0x7fd5280cf430 2024-09-17T03:14:10.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.679+0000 7fd52cca4640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd5280d3720 con 0x7fd5280cf020 2024-09-17T03:14:10.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.680+0000 7fd526d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd52815c5f0 0x7fd528149100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:10.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.680+0000 7fd526d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd52815c5f0 0x7fd528149100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50194/0 (socket says 172.21.15.87:50194) 2024-09-17T03:14:10.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.680+0000 7fd526d9d640 1 -- 172.21.15.87:0/1156527074 learned_addr learned my addr 172.21.15.87:0/1156527074 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:10.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.680+0000 7fd525d9b640 1 --2- 172.21.15.87:0/1156527074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5280cf430 0x7fd5281f9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:10.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.680+0000 7fd52659c640 1 --2- 172.21.15.87:0/1156527074 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5280cf020 0x7fd5280cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:10.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.680+0000 7fd526d9d640 1 -- 172.21.15.87:0/1156527074 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5280cf020 msgr2=0x7fd5280cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:10.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.680+0000 7fd526d9d640 1 --2- 172.21.15.87:0/1156527074 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5280cf020 0x7fd5280cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:10.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.680+0000 7fd526d9d640 1 -- 172.21.15.87:0/1156527074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5280cf430 msgr2=0x7fd5281f9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:10.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.680+0000 7fd526d9d640 1 --2- 172.21.15.87:0/1156527074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5280cf430 0x7fd5281f9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:10.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.680+0000 7fd526d9d640 1 -- 172.21.15.87:0/1156527074 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5281f5c60 con 0x7fd52815c5f0 2024-09-17T03:14:10.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.680+0000 7fd526d9d640 1 --2- 172.21.15.87:0/1156527074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd52815c5f0 0x7fd528149100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd52000c730 tx=0x7fd52000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:10.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.680+0000 7fd516ffd640 1 -- 172.21.15.87:0/1156527074 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd520003b70 con 0x7fd52815c5f0 2024-09-17T03:14:10.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.680+0000 7fd516ffd640 1 -- 172.21.15.87:0/1156527074 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd520003d30 con 0x7fd52815c5f0 2024-09-17T03:14:10.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.680+0000 7fd52cca4640 1 -- 172.21.15.87:0/1156527074 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5281f9fa0 con 0x7fd52815c5f0 2024-09-17T03:14:10.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.680+0000 7fd516ffd640 1 -- 172.21.15.87:0/1156527074 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5200052d0 con 0x7fd52815c5f0 2024-09-17T03:14:10.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.680+0000 7fd52cca4640 1 -- 172.21.15.87:0/1156527074 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5281f6220 con 0x7fd52815c5f0 2024-09-17T03:14:10.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.681+0000 7fd52cca4640 1 -- 172.21.15.87:0/1156527074 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5280b6ad0 con 0x7fd52815c5f0 2024-09-17T03:14:10.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.682+0000 7fd516ffd640 1 -- 172.21.15.87:0/1156527074 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd520005470 con 0x7fd52815c5f0 2024-09-17T03:14:10.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.683+0000 7fd516ffd640 1 --2- 172.21.15.87:0/1156527074 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd508078b90 0x7fd50807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:10.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.683+0000 7fd516ffd640 1 -- 172.21.15.87:0/1156527074 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fd52009a630 con 0x7fd52815c5f0 2024-09-17T03:14:10.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.683+0000 7fd52659c640 1 --2- 172.21.15.87:0/1156527074 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd508078b90 0x7fd50807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:10.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.683+0000 7fd52659c640 1 --2- 172.21.15.87:0/1156527074 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd508078b90 0x7fd50807b0a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd518000f60 tx=0x7fd518009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:10.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.687+0000 7fd516ffd640 1 -- 172.21.15.87:0/1156527074 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd520062770 con 0x7fd52815c5f0 2024-09-17T03:14:10.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.893+0000 7fd52cca4640 1 -- 172.21.15.87:0/1156527074 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5280b5410 con 0x7fd52815c5f0 2024-09-17T03:14:10.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.894+0000 7fd516ffd640 1 -- 172.21.15.87:0/1156527074 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3906 (secure 0 0 0) 0x7fd5200671a0 con 0x7fd52815c5f0 2024-09-17T03:14:10.894 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:10.894 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":3,"btime":"2024-09-17T03:14:09:283379+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4289,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/110106418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":110106418},{"type":"v1","addr":"172.21.15.149:6835","nonce":110106418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1336626873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1336626873},{"type":"v1","addr":"172.21.15.149:6837","nonce":1336626873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/666860906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":666860906},{"type":"v1","addr":"172.21.15.87:6833","nonce":666860906}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4398,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1831073338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1831073338},{"type":"v1","addr":"172.21.15.87:6835","nonce":1831073338}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-17T03:14:10.894 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 3 2024-09-17T03:14:10.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.896+0000 7fd52cca4640 1 -- 172.21.15.87:0/1156527074 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd508078b90 msgr2=0x7fd50807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:10.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.896+0000 7fd52cca4640 1 --2- 172.21.15.87:0/1156527074 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd508078b90 0x7fd50807b0a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd518000f60 tx=0x7fd518009040 comp rx=0 tx=0).stop 2024-09-17T03:14:10.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.896+0000 7fd52cca4640 1 -- 172.21.15.87:0/1156527074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd52815c5f0 msgr2=0x7fd528149100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:10.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.896+0000 7fd52cca4640 1 --2- 172.21.15.87:0/1156527074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd52815c5f0 0x7fd528149100 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd52000c730 tx=0x7fd52000cbf0 comp rx=0 tx=0).stop 2024-09-17T03:14:10.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.896+0000 7fd52cca4640 1 -- 172.21.15.87:0/1156527074 shutdown_connections 2024-09-17T03:14:10.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.896+0000 7fd52cca4640 1 --2- 172.21.15.87:0/1156527074 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd508078b90 0x7fd50807b0a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:10.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.896+0000 7fd52cca4640 1 --2- 172.21.15.87:0/1156527074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd52815c5f0 0x7fd528149100 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:10.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.896+0000 7fd52cca4640 1 --2- 172.21.15.87:0/1156527074 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd5280cf430 0x7fd5281f9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:10.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.896+0000 7fd52cca4640 1 --2- 172.21.15.87:0/1156527074 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd5280cf020 0x7fd5280cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:10.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.896+0000 7fd52cca4640 1 -- 172.21.15.87:0/1156527074 >> 172.21.15.87:0/1156527074 conn(0x7fd528069900 msgr2=0x7fd5280b4680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:10.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.896+0000 7fd52cca4640 1 -- 172.21.15.87:0/1156527074 shutdown_connections 2024-09-17T03:14:10.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:10.896+0000 7fd52cca4640 1 -- 172.21.15.87:0/1156527074 wait complete. 2024-09-17T03:14:10.907 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T03:14:11.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.018+0000 7f5d4f1e6640 1 -- 172.21.15.87:0/2933104804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d480ce7c0 msgr2=0x7f5d480cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:11.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.018+0000 7f5d4f1e6640 1 --2- 172.21.15.87:0/2933104804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d480ce7c0 0x7f5d480cebd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5d480d1870 tx=0x7f5d3c02a560 comp rx=0 tx=0).stop 2024-09-17T03:14:11.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.018+0000 7f5d4f1e6640 1 -- 172.21.15.87:0/2933104804 shutdown_connections 2024-09-17T03:14:11.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.018+0000 7f5d4f1e6640 1 --2- 172.21.15.87:0/2933104804 >> v2:172.21.15.149:3300/0 conn(0x7f5d48070d20 0x7f5d48071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:11.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.018+0000 7f5d4f1e6640 1 --2- 172.21.15.87:0/2933104804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d480ce7c0 0x7f5d480cebd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:11.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.018+0000 7f5d4f1e6640 1 -- 172.21.15.87:0/2933104804 >> 172.21.15.87:0/2933104804 conn(0x7f5d480d08c0 msgr2=0x7f5d480d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:11.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.018+0000 7f5d4f1e6640 1 -- 172.21.15.87:0/2933104804 shutdown_connections 2024-09-17T03:14:11.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.018+0000 7f5d4f1e6640 1 -- 172.21.15.87:0/2933104804 wait complete. 2024-09-17T03:14:11.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.020+0000 7f5d4f1e6640 1 Processor -- start 2024-09-17T03:14:11.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.020+0000 7f5d4f1e6640 1 -- start start 2024-09-17T03:14:11.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.021+0000 7f5d4f1e6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d48068d00 0x7f5d481f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:11.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.021+0000 7f5d4f1e6640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d48070d20 0x7f5d48152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:11.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.021+0000 7f5d4f1e6640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5d480ce7c0 0x7f5d48151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:11.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.021+0000 7f5d4f1e6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5d48071670 con 0x7f5d48068d00 2024-09-17T03:14:11.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.021+0000 7f5d4f1e6640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5d4815bab0 con 0x7f5d48070d20 2024-09-17T03:14:11.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.021+0000 7f5d4f1e6640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5d480717f0 con 0x7f5d480ce7c0 2024-09-17T03:14:11.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.021+0000 7f5d4cf82640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d48068d00 0x7f5d481f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:11.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.021+0000 7f5d4cf82640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d48068d00 0x7f5d481f34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50208/0 (socket says 172.21.15.87:50208) 2024-09-17T03:14:11.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.021+0000 7f5d4cf82640 1 -- 172.21.15.87:0/266588528 learned_addr learned my addr 172.21.15.87:0/266588528 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:11.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.021+0000 7f5d47fff640 1 --2- 172.21.15.87:0/266588528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d48070d20 0x7f5d48152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:11.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.021+0000 7f5d4d783640 1 --2- 172.21.15.87:0/266588528 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5d480ce7c0 0x7f5d48151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:11.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.022+0000 7f5d4cf82640 1 -- 172.21.15.87:0/266588528 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5d480ce7c0 msgr2=0x7f5d48151910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:11.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.022+0000 7f5d4cf82640 1 --2- 172.21.15.87:0/266588528 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5d480ce7c0 0x7f5d48151910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:11.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.022+0000 7f5d4cf82640 1 -- 172.21.15.87:0/266588528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d48070d20 msgr2=0x7f5d48152e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:11.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.022+0000 7f5d4cf82640 1 --2- 172.21.15.87:0/266588528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d48070d20 0x7f5d48152e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:11.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.022+0000 7f5d4cf82640 1 -- 172.21.15.87:0/266588528 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d48151e50 con 0x7f5d48068d00 2024-09-17T03:14:11.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.022+0000 7f5d4d783640 1 --2- 172.21.15.87:0/266588528 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5d480ce7c0 0x7f5d48151910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:11.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.022+0000 7f5d4cf82640 1 --2- 172.21.15.87:0/266588528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d48068d00 0x7f5d481f34e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800b420 tx=0x7f5d3800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:11.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.022+0000 7f5d457fa640 1 -- 172.21.15.87:0/266588528 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38011830 con 0x7f5d48068d00 2024-09-17T03:14:11.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.022+0000 7f5d457fa640 1 -- 172.21.15.87:0/266588528 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d38016070 con 0x7f5d48068d00 2024-09-17T03:14:11.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.022+0000 7f5d457fa640 1 -- 172.21.15.87:0/266588528 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d38011d60 con 0x7f5d48068d00 2024-09-17T03:14:11.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.022+0000 7f5d4f1e6640 1 -- 172.21.15.87:0/266588528 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d481f3d20 con 0x7f5d48068d00 2024-09-17T03:14:11.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.022+0000 7f5d4f1e6640 1 -- 172.21.15.87:0/266588528 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d481f42e0 con 0x7f5d48068d00 2024-09-17T03:14:11.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.023+0000 7f5d4f1e6640 1 -- 172.21.15.87:0/266588528 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d48071130 con 0x7f5d48068d00 2024-09-17T03:14:11.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.024+0000 7f5d457fa640 1 -- 172.21.15.87:0/266588528 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5d3801b460 con 0x7f5d48068d00 2024-09-17T03:14:11.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.024+0000 7f5d457fa640 1 --2- 172.21.15.87:0/266588528 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5d34078bc0 0x7f5d3407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:11.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.024+0000 7f5d457fa640 1 -- 172.21.15.87:0/266588528 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f5d3809a2b0 con 0x7f5d48068d00 2024-09-17T03:14:11.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.024+0000 7f5d47fff640 1 --2- 172.21.15.87:0/266588528 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5d34078bc0 0x7f5d3407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:11.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.025+0000 7f5d47fff640 1 --2- 172.21.15.87:0/266588528 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5d34078bc0 0x7f5d3407b0d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c033000 tx=0x7f5d3c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:11.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.029+0000 7f5d457fa640 1 -- 172.21.15.87:0/266588528 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f5d38020020 con 0x7f5d48068d00 2024-09-17T03:14:11.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.203+0000 7f5d4f1e6640 1 -- 172.21.15.87:0/266588528 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5d480cd0b0 con 0x7f5d48068d00 2024-09-17T03:14:11.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.203+0000 7f5d457fa640 1 -- 172.21.15.87:0/266588528 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) ==== 84+0+65 (secure 0 0 0) 0x7f5d38062b60 con 0x7f5d48068d00 2024-09-17T03:14:11.203 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:11.203 INFO:teuthology.orchestra.run.smithi087.stdout:[ 2024-09-17T03:14:11.204 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:11.204 INFO:teuthology.orchestra.run.smithi087.stdout: "poolnum": 1, 2024-09-17T03:14:11.204 INFO:teuthology.orchestra.run.smithi087.stdout: "poolname": ".mgr" 2024-09-17T03:14:11.204 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:11.204 INFO:teuthology.orchestra.run.smithi087.stdout:] 2024-09-17T03:14:11.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.205+0000 7f5d4f1e6640 1 -- 172.21.15.87:0/266588528 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5d34078bc0 msgr2=0x7f5d3407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:11.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.205+0000 7f5d4f1e6640 1 --2- 172.21.15.87:0/266588528 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5d34078bc0 0x7f5d3407b0d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c033000 tx=0x7f5d3c004730 comp rx=0 tx=0).stop 2024-09-17T03:14:11.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.205+0000 7f5d4f1e6640 1 -- 172.21.15.87:0/266588528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d48068d00 msgr2=0x7f5d481f34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:11.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.205+0000 7f5d4f1e6640 1 --2- 172.21.15.87:0/266588528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d48068d00 0x7f5d481f34e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800b420 tx=0x7f5d3800b8e0 comp rx=0 tx=0).stop 2024-09-17T03:14:11.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.206+0000 7f5d4f1e6640 1 -- 172.21.15.87:0/266588528 shutdown_connections 2024-09-17T03:14:11.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.206+0000 7f5d4f1e6640 1 --2- 172.21.15.87:0/266588528 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5d34078bc0 0x7f5d3407b0d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:11.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.206+0000 7f5d4f1e6640 1 --2- 172.21.15.87:0/266588528 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5d480ce7c0 0x7f5d48151910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:11.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.206+0000 7f5d4f1e6640 1 --2- 172.21.15.87:0/266588528 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5d48070d20 0x7f5d48152e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:11.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.206+0000 7f5d4f1e6640 1 --2- 172.21.15.87:0/266588528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d48068d00 0x7f5d481f34e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:11.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.206+0000 7f5d4f1e6640 1 -- 172.21.15.87:0/266588528 >> 172.21.15.87:0/266588528 conn(0x7f5d480d08c0 msgr2=0x7f5d48157df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:11.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.206+0000 7f5d4f1e6640 1 -- 172.21.15.87:0/266588528 shutdown_connections 2024-09-17T03:14:11.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.206+0000 7f5d4f1e6640 1 -- 172.21.15.87:0/266588528 wait complete. 2024-09-17T03:14:11.216 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T03:14:11.216 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T03:14:11.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.334+0000 7f07ba1b0640 1 -- 172.21.15.87:0/3848534422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07b40b9ae0 msgr2=0x7f07b4160390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:11.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.334+0000 7f07ba1b0640 1 --2- 172.21.15.87:0/3848534422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07b40b9ae0 0x7f07b4160390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f07a40098e0 tx=0x7f07a402a530 comp rx=0 tx=0).stop 2024-09-17T03:14:11.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.334+0000 7f07ba1b0640 1 -- 172.21.15.87:0/3848534422 shutdown_connections 2024-09-17T03:14:11.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.334+0000 7f07ba1b0640 1 --2- 172.21.15.87:0/3848534422 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f07b40cfaa0 0x7f07b4162590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:11.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.334+0000 7f07ba1b0640 1 --2- 172.21.15.87:0/3848534422 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07b40b9ae0 0x7f07b4160390 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:11.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.334+0000 7f07ba1b0640 1 -- 172.21.15.87:0/3848534422 >> 172.21.15.87:0/3848534422 conn(0x7f07b40d4510 msgr2=0x7f07b40d4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:11.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.334+0000 7f07ba1b0640 1 -- 172.21.15.87:0/3848534422 shutdown_connections 2024-09-17T03:14:11.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.334+0000 7f07ba1b0640 1 -- 172.21.15.87:0/3848534422 wait complete. 2024-09-17T03:14:11.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.337+0000 7f07ba1b0640 1 Processor -- start 2024-09-17T03:14:11.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.337+0000 7f07ba1b0640 1 -- start start 2024-09-17T03:14:11.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.337+0000 7f07ba1b0640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f07b40b33c0 0x7f07b40c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:11.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.337+0000 7f07ba1b0640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07b40b9ae0 0x7f07b40b68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:11.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.337+0000 7f07ba1b0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07b40cfaa0 0x7f07b40cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:11.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.337+0000 7f07ba1b0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f07b4162e60 con 0x7f07b40cfaa0 2024-09-17T03:14:11.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.337+0000 7f07ba1b0640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f07b4162ce0 con 0x7f07b40b9ae0 2024-09-17T03:14:11.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.337+0000 7f07ba1b0640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f07b4162fe0 con 0x7f07b40b33c0 2024-09-17T03:14:11.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.337+0000 7f07b3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07b40cfaa0 0x7f07b40cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:11.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.337+0000 7f07b3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07b40cfaa0 0x7f07b40cac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50224/0 (socket says 172.21.15.87:50224) 2024-09-17T03:14:11.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.337+0000 7f07b3fff640 1 -- 172.21.15.87:0/3537270662 learned_addr learned my addr 172.21.15.87:0/3537270662 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:11.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.337+0000 7f07b37fe640 1 --2- 172.21.15.87:0/3537270662 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f07b40b33c0 0x7f07b40c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:11.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.337+0000 7f07b2ffd640 1 --2- 172.21.15.87:0/3537270662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07b40b9ae0 0x7f07b40b68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:11.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.338+0000 7f07b3fff640 1 -- 172.21.15.87:0/3537270662 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f07b40b33c0 msgr2=0x7f07b40c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:11.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.338+0000 7f07b3fff640 1 --2- 172.21.15.87:0/3537270662 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f07b40b33c0 0x7f07b40c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:11.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.338+0000 7f07b3fff640 1 -- 172.21.15.87:0/3537270662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07b40b9ae0 msgr2=0x7f07b40b68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:11.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.338+0000 7f07b3fff640 1 --2- 172.21.15.87:0/3537270662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07b40b9ae0 0x7f07b40b68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:11.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.338+0000 7f07b3fff640 1 -- 172.21.15.87:0/3537270662 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b40b6e30 con 0x7f07b40cfaa0 2024-09-17T03:14:11.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.338+0000 7f07b3fff640 1 --2- 172.21.15.87:0/3537270662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07b40cfaa0 0x7f07b40cac50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00ea40 tx=0x7f07ac00ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:11.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.338+0000 7f079bfff640 1 -- 172.21.15.87:0/3537270662 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac00ce40 con 0x7f07b40cfaa0 2024-09-17T03:14:11.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.338+0000 7f07ba1b0640 1 -- 172.21.15.87:0/3537270662 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b4154120 con 0x7f07b40cfaa0 2024-09-17T03:14:11.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.338+0000 7f079bfff640 1 -- 172.21.15.87:0/3537270662 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07ac004c10 con 0x7f07b40cfaa0 2024-09-17T03:14:11.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.338+0000 7f079bfff640 1 -- 172.21.15.87:0/3537270662 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac005020 con 0x7f07b40cfaa0 2024-09-17T03:14:11.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.338+0000 7f07ba1b0640 1 -- 172.21.15.87:0/3537270662 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b40cbd10 con 0x7f07b40cfaa0 2024-09-17T03:14:11.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.339+0000 7f07ba1b0640 1 -- 172.21.15.87:0/3537270662 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07b40cb7a0 con 0x7f07b40cfaa0 2024-09-17T03:14:11.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.340+0000 7f079bfff640 1 -- 172.21.15.87:0/3537270662 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f07ac0051c0 con 0x7f07b40cfaa0 2024-09-17T03:14:11.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.340+0000 7f079bfff640 1 --2- 172.21.15.87:0/3537270662 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f07a0078b90 0x7f07a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:11.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.340+0000 7f079bfff640 1 -- 172.21.15.87:0/3537270662 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f07ac09b750 con 0x7f07b40cfaa0 2024-09-17T03:14:11.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.340+0000 7f07b37fe640 1 --2- 172.21.15.87:0/3537270662 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f07a0078b90 0x7f07a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:11.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.341+0000 7f07b37fe640 1 --2- 172.21.15.87:0/3537270662 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f07a0078b90 0x7f07a007b0a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f07a4034000 tx=0x7f07a4004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:11.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.345+0000 7f079bfff640 1 -- 172.21.15.87:0/3537270662 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f07b40b6e30 con 0x7f07b40cfaa0 2024-09-17T03:14:11.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:11.520+0000 7f07ba1b0640 1 -- 172.21.15.87:0/3537270662 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f07b4160390 con 0x7f07b40cfaa0 2024-09-17T03:14:12.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.298+0000 7f079bfff640 1 -- 172.21.15.87:0/3537270662 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) ==== 138+0+0 (secure 0 0 0) 0x7f07ac063810 con 0x7f07b40cfaa0 2024-09-17T03:14:12.297 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' created 2024-09-17T03:14:12.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.300+0000 7f07ba1b0640 1 -- 172.21.15.87:0/3537270662 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f07a0078b90 msgr2=0x7f07a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:12.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.300+0000 7f07ba1b0640 1 --2- 172.21.15.87:0/3537270662 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f07a0078b90 0x7f07a007b0a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f07a4034000 tx=0x7f07a4004730 comp rx=0 tx=0).stop 2024-09-17T03:14:12.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.300+0000 7f07ba1b0640 1 -- 172.21.15.87:0/3537270662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07b40cfaa0 msgr2=0x7f07b40cac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:12.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.300+0000 7f07ba1b0640 1 --2- 172.21.15.87:0/3537270662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07b40cfaa0 0x7f07b40cac50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f07ac00ea40 tx=0x7f07ac00ef00 comp rx=0 tx=0).stop 2024-09-17T03:14:12.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.301+0000 7f07ba1b0640 1 -- 172.21.15.87:0/3537270662 shutdown_connections 2024-09-17T03:14:12.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.301+0000 7f07ba1b0640 1 --2- 172.21.15.87:0/3537270662 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f07a0078b90 0x7f07a007b0a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:12.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.301+0000 7f07ba1b0640 1 --2- 172.21.15.87:0/3537270662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07b40cfaa0 0x7f07b40cac50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:12.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.301+0000 7f07ba1b0640 1 --2- 172.21.15.87:0/3537270662 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f07b40b9ae0 0x7f07b40b68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:12.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.301+0000 7f07ba1b0640 1 --2- 172.21.15.87:0/3537270662 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f07b40b33c0 0x7f07b40c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:12.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.301+0000 7f07ba1b0640 1 -- 172.21.15.87:0/3537270662 >> 172.21.15.87:0/3537270662 conn(0x7f07b40d4510 msgr2=0x7f07b4160aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:12.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.301+0000 7f07ba1b0640 1 -- 172.21.15.87:0/3537270662 shutdown_connections 2024-09-17T03:14:12.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.301+0000 7f07ba1b0640 1 -- 172.21.15.87:0/3537270662 wait complete. 2024-09-17T03:14:12.314 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T03:14:12.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.437+0000 7f44a184d640 1 -- 172.21.15.87:0/341643243 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f449c0d4350 msgr2=0x7f449c0d4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:12.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.437+0000 7f44a184d640 1 --2- 172.21.15.87:0/341643243 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f449c0d4350 0x7f449c0d4760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4494005fd0 tx=0x7f449402aa90 comp rx=0 tx=0).stop 2024-09-17T03:14:12.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.437+0000 7f44a184d640 1 -- 172.21.15.87:0/341643243 shutdown_connections 2024-09-17T03:14:12.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.437+0000 7f44a184d640 1 --2- 172.21.15.87:0/341643243 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f449c0d4350 0x7f449c0d4760 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:12.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.437+0000 7f44a184d640 1 --2- 172.21.15.87:0/341643243 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f449c0b3080 0x7f449c0cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:12.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.437+0000 7f44a184d640 1 -- 172.21.15.87:0/341643243 >> 172.21.15.87:0/341643243 conn(0x7f449c08bfc0 msgr2=0x7f449c09d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:12.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.437+0000 7f44a184d640 1 -- 172.21.15.87:0/341643243 shutdown_connections 2024-09-17T03:14:12.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.437+0000 7f44a184d640 1 -- 172.21.15.87:0/341643243 wait complete. 2024-09-17T03:14:12.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.438+0000 7f44a184d640 1 Processor -- start 2024-09-17T03:14:12.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.438+0000 7f44a184d640 1 -- start start 2024-09-17T03:14:12.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.440+0000 7f44a184d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f449c0b3080 0x7f449c0a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:12.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.440+0000 7f44a184d640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f449c0b3e00 0x7f449c1f9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:12.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.440+0000 7f44a184d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f449c0d4350 0x7f449c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:12.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.440+0000 7f44a184d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f449c0d29e0 con 0x7f449c0b3080 2024-09-17T03:14:12.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.440+0000 7f44a184d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f449c0d2860 con 0x7f449c0d4350 2024-09-17T03:14:12.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.440+0000 7f44a184d640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f449c0d2b60 con 0x7f449c0b3e00 2024-09-17T03:14:12.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.440+0000 7f449affd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f449c0b3080 0x7f449c0a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:12.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.440+0000 7f449affd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f449c0b3080 0x7f449c0a10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50250/0 (socket says 172.21.15.87:50250) 2024-09-17T03:14:12.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.440+0000 7f449affd640 1 -- 172.21.15.87:0/2621813797 learned_addr learned my addr 172.21.15.87:0/2621813797 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:12.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.440+0000 7f449affd640 1 -- 172.21.15.87:0/2621813797 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f449c0b3e00 msgr2=0x7f449c1f9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:12.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.440+0000 7f449b7fe640 1 --2- 172.21.15.87:0/2621813797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f449c0d4350 0x7f449c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:12.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.440+0000 7f4493fff640 1 --2- 172.21.15.87:0/2621813797 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f449c0b3e00 0x7f449c1f9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:12.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.440+0000 7f449affd640 1 --2- 172.21.15.87:0/2621813797 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f449c0b3e00 0x7f449c1f9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:12.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.440+0000 7f449affd640 1 -- 172.21.15.87:0/2621813797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f449c0d4350 msgr2=0x7f449c0ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:12.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.440+0000 7f449affd640 1 --2- 172.21.15.87:0/2621813797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f449c0d4350 0x7f449c0ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:12.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.440+0000 7f449affd640 1 -- 172.21.15.87:0/2621813797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f449c0a1a00 con 0x7f449c0b3080 2024-09-17T03:14:12.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.440+0000 7f449b7fe640 1 --2- 172.21.15.87:0/2621813797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f449c0d4350 0x7f449c0ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:14:12.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.440+0000 7f449affd640 1 --2- 172.21.15.87:0/2621813797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f449c0b3080 0x7f449c0a10f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f448c009fd0 tx=0x7f448c00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:12.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.440+0000 7f4498ff9640 1 -- 172.21.15.87:0/2621813797 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c00ccc0 con 0x7f449c0b3080 2024-09-17T03:14:12.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.440+0000 7f4498ff9640 1 -- 172.21.15.87:0/2621813797 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f448c00ee70 con 0x7f449c0b3080 2024-09-17T03:14:12.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.440+0000 7f44a184d640 1 -- 172.21.15.87:0/2621813797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f449c0a1ce0 con 0x7f449c0b3080 2024-09-17T03:14:12.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.440+0000 7f4498ff9640 1 -- 172.21.15.87:0/2621813797 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c00ce60 con 0x7f449c0b3080 2024-09-17T03:14:12.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.440+0000 7f44a184d640 1 -- 172.21.15.87:0/2621813797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f449c1f9840 con 0x7f449c0b3080 2024-09-17T03:14:12.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.442+0000 7f44a184d640 1 -- 172.21.15.87:0/2621813797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f449c0d2fe0 con 0x7f449c0b3080 2024-09-17T03:14:12.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.442+0000 7f4498ff9640 1 -- 172.21.15.87:0/2621813797 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f448c004280 con 0x7f449c0b3080 2024-09-17T03:14:12.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.443+0000 7f4498ff9640 1 --2- 172.21.15.87:0/2621813797 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f447c078bc0 0x7f447c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:12.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.443+0000 7f4498ff9640 1 -- 172.21.15.87:0/2621813797 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5227+0+0 (secure 0 0 0) 0x7f448c09b4b0 con 0x7f449c0b3080 2024-09-17T03:14:12.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.443+0000 7f4493fff640 1 --2- 172.21.15.87:0/2621813797 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f447c078bc0 0x7f447c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:12.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.443+0000 7f4493fff640 1 --2- 172.21.15.87:0/2621813797 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f447c078bc0 0x7f447c07b0d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f449c06fd20 tx=0x7f4484009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:12.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.448+0000 7f4498ff9640 1 -- 172.21.15.87:0/2621813797 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f448c063ad0 con 0x7f449c0b3080 2024-09-17T03:14:12.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:12.622+0000 7f44a184d640 1 -- 172.21.15.87:0/2621813797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f449c0a2150 con 0x7f449c0b3080 2024-09-17T03:14:13.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.300+0000 7f4498ff9640 1 -- 172.21.15.87:0/2621813797 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) ==== 170+0+0 (secure 0 0 0) 0x7f448c0633d0 con 0x7f449c0b3080 2024-09-17T03:14:13.299 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' created 2024-09-17T03:14:13.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.303+0000 7f44a184d640 1 -- 172.21.15.87:0/2621813797 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f447c078bc0 msgr2=0x7f447c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:13.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.303+0000 7f44a184d640 1 --2- 172.21.15.87:0/2621813797 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f447c078bc0 0x7f447c07b0d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f449c06fd20 tx=0x7f4484009040 comp rx=0 tx=0).stop 2024-09-17T03:14:13.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.303+0000 7f44a184d640 1 -- 172.21.15.87:0/2621813797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f449c0b3080 msgr2=0x7f449c0a10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:13.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.303+0000 7f44a184d640 1 --2- 172.21.15.87:0/2621813797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f449c0b3080 0x7f449c0a10f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f448c009fd0 tx=0x7f448c00ec20 comp rx=0 tx=0).stop 2024-09-17T03:14:13.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.303+0000 7f44a184d640 1 -- 172.21.15.87:0/2621813797 shutdown_connections 2024-09-17T03:14:13.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.303+0000 7f44a184d640 1 --2- 172.21.15.87:0/2621813797 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f447c078bc0 0x7f447c07b0d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:13.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.303+0000 7f44a184d640 1 --2- 172.21.15.87:0/2621813797 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f449c0d4350 0x7f449c0ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:13.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.303+0000 7f44a184d640 1 --2- 172.21.15.87:0/2621813797 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f449c0b3e00 0x7f449c1f9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:13.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.303+0000 7f44a184d640 1 --2- 172.21.15.87:0/2621813797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f449c0b3080 0x7f449c0a10f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:13.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.303+0000 7f44a184d640 1 -- 172.21.15.87:0/2621813797 >> 172.21.15.87:0/2621813797 conn(0x7f449c08bfc0 msgr2=0x7f449c06f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:13.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.303+0000 7f44a184d640 1 -- 172.21.15.87:0/2621813797 shutdown_connections 2024-09-17T03:14:13.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.303+0000 7f44a184d640 1 -- 172.21.15.87:0/2621813797 wait complete. 2024-09-17T03:14:13.329 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T03:14:13.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.449+0000 7f30ba269640 1 -- 172.21.15.87:0/1411443265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b40b3fb0 msgr2=0x7f30b4162e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:13.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.449+0000 7f30ba269640 1 --2- 172.21.15.87:0/1411443265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b40b3fb0 0x7f30b4162e30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f30b40b3b20 tx=0x7f30a402a410 comp rx=0 tx=0).stop 2024-09-17T03:14:13.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.449+0000 7f30ba269640 1 -- 172.21.15.87:0/1411443265 shutdown_connections 2024-09-17T03:14:13.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.449+0000 7f30ba269640 1 --2- 172.21.15.87:0/1411443265 >> v2:172.21.15.149:3300/0 conn(0x7f30b40b31f0 0x7f30b40d43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:13.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.449+0000 7f30ba269640 1 --2- 172.21.15.87:0/1411443265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b40b3fb0 0x7f30b4162e30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:13.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.449+0000 7f30ba269640 1 -- 172.21.15.87:0/1411443265 >> 172.21.15.87:0/1411443265 conn(0x7f30b408bfc0 msgr2=0x7f30b409d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:13.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.449+0000 7f30ba269640 1 -- 172.21.15.87:0/1411443265 shutdown_connections 2024-09-17T03:14:13.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.449+0000 7f30ba269640 1 -- 172.21.15.87:0/1411443265 wait complete. 2024-09-17T03:14:13.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.450+0000 7f30ba269640 1 Processor -- start 2024-09-17T03:14:13.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.451+0000 7f30ba269640 1 -- start start 2024-09-17T03:14:13.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.452+0000 7f30ba269640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30b40b31f0 0x7f30b409f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:13.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.452+0000 7f30ba269640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30b40b3fb0 0x7f30b41ea230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:13.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.452+0000 7f30ba269640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b40d4920 0x7f30b4150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:13.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.452+0000 7f30ba269640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f30b41634f0 con 0x7f30b40d4920 2024-09-17T03:14:13.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.452+0000 7f30ba269640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f30b4163370 con 0x7f30b40b31f0 2024-09-17T03:14:13.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.452+0000 7f30ba269640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f30b40cf6c0 con 0x7f30b40b3fb0 2024-09-17T03:14:13.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.452+0000 7f30b8806640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b40d4920 0x7f30b4150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:13.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.452+0000 7f30b8806640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b40d4920 0x7f30b4150980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50272/0 (socket says 172.21.15.87:50272) 2024-09-17T03:14:13.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.452+0000 7f30b8806640 1 -- 172.21.15.87:0/3171194978 learned_addr learned my addr 172.21.15.87:0/3171194978 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:13.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.452+0000 7f30b3fff640 1 --2- 172.21.15.87:0/3171194978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30b40b31f0 0x7f30b409f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:13.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.452+0000 7f30b37fe640 1 --2- 172.21.15.87:0/3171194978 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30b40b3fb0 0x7f30b41ea230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:13.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.452+0000 7f30b8806640 1 -- 172.21.15.87:0/3171194978 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30b40b3fb0 msgr2=0x7f30b41ea230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:13.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.452+0000 7f30b8806640 1 --2- 172.21.15.87:0/3171194978 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30b40b3fb0 0x7f30b41ea230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:13.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.452+0000 7f30b8806640 1 -- 172.21.15.87:0/3171194978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30b40b31f0 msgr2=0x7f30b409f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:13.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.452+0000 7f30b8806640 1 --2- 172.21.15.87:0/3171194978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30b40b31f0 0x7f30b409f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:13.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.452+0000 7f30b8806640 1 -- 172.21.15.87:0/3171194978 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b41fa4b0 con 0x7f30b40d4920 2024-09-17T03:14:13.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.453+0000 7f30b8806640 1 --2- 172.21.15.87:0/3171194978 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b40d4920 0x7f30b4150980 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00cb10 tx=0x7f30ac00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:13.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.453+0000 7f30b0ff9640 1 -- 172.21.15.87:0/3171194978 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac016020 con 0x7f30b40d4920 2024-09-17T03:14:13.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.453+0000 7f30b0ff9640 1 -- 172.21.15.87:0/3171194978 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30ac00ad60 con 0x7f30b40d4920 2024-09-17T03:14:13.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.453+0000 7f30b0ff9640 1 -- 172.21.15.87:0/3171194978 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30ac012680 con 0x7f30b40d4920 2024-09-17T03:14:13.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.453+0000 7f30ba269640 1 -- 172.21.15.87:0/3171194978 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30b41e8420 con 0x7f30b40d4920 2024-09-17T03:14:13.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.453+0000 7f30ba269640 1 -- 172.21.15.87:0/3171194978 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30b41e89e0 con 0x7f30b40d4920 2024-09-17T03:14:13.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.454+0000 7f30ba269640 1 -- 172.21.15.87:0/3171194978 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30b40d43e0 con 0x7f30b40d4920 2024-09-17T03:14:13.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.454+0000 7f30b0ff9640 1 -- 172.21.15.87:0/3171194978 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f30ac019020 con 0x7f30b40d4920 2024-09-17T03:14:13.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.455+0000 7f30b0ff9640 1 --2- 172.21.15.87:0/3171194978 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f30a0078b90 0x7f30a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:13.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.455+0000 7f30b0ff9640 1 -- 172.21.15.87:0/3171194978 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5609+0+0 (secure 0 0 0) 0x7f30ac09af20 con 0x7f30b40d4920 2024-09-17T03:14:13.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.455+0000 7f30b3fff640 1 --2- 172.21.15.87:0/3171194978 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f30a0078b90 0x7f30a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:13.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.456+0000 7f30b3fff640 1 --2- 172.21.15.87:0/3171194978 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f30a0078b90 0x7f30a007b0a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f30a4034000 tx=0x7f30a4004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:13.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.460+0000 7f30b0ff9640 1 -- 172.21.15.87:0/3171194978 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f30ac020030 con 0x7f30b40d4920 2024-09-17T03:14:13.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:13.671+0000 7f30ba269640 1 -- 172.21.15.87:0/3171194978 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f30b40d0c90 con 0x7f30b40d4920 2024-09-17T03:14:14.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.314+0000 7f30b0ff9640 1 -- 172.21.15.87:0/3171194978 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f30ac062d70 con 0x7f30b40d4920 2024-09-17T03:14:14.313 INFO:teuthology.orchestra.run.smithi087.stderr:new fs with metadata pool 2 and data pool 3 2024-09-17T03:14:14.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.317+0000 7f30ba269640 1 -- 172.21.15.87:0/3171194978 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f30a0078b90 msgr2=0x7f30a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:14.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.317+0000 7f30ba269640 1 --2- 172.21.15.87:0/3171194978 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f30a0078b90 0x7f30a007b0a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f30a4034000 tx=0x7f30a4004730 comp rx=0 tx=0).stop 2024-09-17T03:14:14.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.317+0000 7f30ba269640 1 -- 172.21.15.87:0/3171194978 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b40d4920 msgr2=0x7f30b4150980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:14.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.317+0000 7f30ba269640 1 --2- 172.21.15.87:0/3171194978 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b40d4920 0x7f30b4150980 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00cb10 tx=0x7f30ac00a060 comp rx=0 tx=0).stop 2024-09-17T03:14:14.316 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.318+0000 7f30ba269640 1 -- 172.21.15.87:0/3171194978 shutdown_connections 2024-09-17T03:14:14.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.318+0000 7f30ba269640 1 --2- 172.21.15.87:0/3171194978 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f30a0078b90 0x7f30a007b0a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:14.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.318+0000 7f30ba269640 1 --2- 172.21.15.87:0/3171194978 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b40d4920 0x7f30b4150980 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:14.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.318+0000 7f30ba269640 1 --2- 172.21.15.87:0/3171194978 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30b40b3fb0 0x7f30b41ea230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:14.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.318+0000 7f30ba269640 1 --2- 172.21.15.87:0/3171194978 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30b40b31f0 0x7f30b409f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:14.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.318+0000 7f30ba269640 1 -- 172.21.15.87:0/3171194978 >> 172.21.15.87:0/3171194978 conn(0x7f30b408bfc0 msgr2=0x7f30b40b1060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:14.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.324+0000 7f30ba269640 1 -- 172.21.15.87:0/3171194978 shutdown_connections 2024-09-17T03:14:14.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.333+0000 7f30ba269640 1 -- 172.21.15.87:0/3171194978 wait complete. 2024-09-17T03:14:14.355 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:14:14.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.470+0000 7f802ede4640 1 -- 172.21.15.87:0/3225477652 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80180041c0 msgr2=0x7f8018004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:14.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.470+0000 7f802ede4640 1 --2- 172.21.15.87:0/3225477652 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80180041c0 0x7f8018004800 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8014001830 tx=0x7f8018005830 comp rx=0 tx=0).stop 2024-09-17T03:14:14.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.470+0000 7f802d381640 1 -- 172.21.15.87:0/3225477652 reap_dead start 2024-09-17T03:14:14.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.470+0000 7f802ede4640 1 -- 172.21.15.87:0/3225477652 shutdown_connections 2024-09-17T03:14:14.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.470+0000 7f802ede4640 1 -- 172.21.15.87:0/3225477652 >> 172.21.15.87:0/3225477652 conn(0x7f8028069360 msgr2=0x7f80280ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:14.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.470+0000 7f802ede4640 1 -- 172.21.15.87:0/3225477652 shutdown_connections 2024-09-17T03:14:14.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.470+0000 7f802ede4640 1 -- 172.21.15.87:0/3225477652 wait complete. 2024-09-17T03:14:14.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.472+0000 7f802ede4640 1 Processor -- start 2024-09-17T03:14:14.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.472+0000 7f802ede4640 1 -- start start 2024-09-17T03:14:14.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.473+0000 7f802ede4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80180041c0 0x7f80280ce170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:14.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.473+0000 7f802ede4640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f80280b7b40 0x7f80280b5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:14.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.473+0000 7f802ede4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80280b5bb0 0x7f80280b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:14.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.473+0000 7f802ede4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f802806def0 con 0x7f80280b5bb0 2024-09-17T03:14:14.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.473+0000 7f802ede4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f802815c690 con 0x7f80180041c0 2024-09-17T03:14:14.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.473+0000 7f802ede4640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f802806e070 con 0x7f80280b7b40 2024-09-17T03:14:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.473+0000 7f802d381640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80280b5bb0 0x7f80280b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.473+0000 7f802d381640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80280b5bb0 0x7f80280b5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50288/0 (socket says 172.21.15.87:50288) 2024-09-17T03:14:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.473+0000 7f802d381640 1 -- 172.21.15.87:0/296909859 learned_addr learned my addr 172.21.15.87:0/296909859 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.473+0000 7f802cb80640 1 --2- 172.21.15.87:0/296909859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80180041c0 0x7f80280ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.473+0000 7f8027fff640 1 --2- 172.21.15.87:0/296909859 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f80280b7b40 0x7f80280b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.473+0000 7f802d381640 1 -- 172.21.15.87:0/296909859 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f80280b7b40 msgr2=0x7f80280b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.473+0000 7f802d381640 1 --2- 172.21.15.87:0/296909859 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f80280b7b40 0x7f80280b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.473+0000 7f802d381640 1 -- 172.21.15.87:0/296909859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80180041c0 msgr2=0x7f80280ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.473+0000 7f802d381640 1 --2- 172.21.15.87:0/296909859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80180041c0 0x7f80280ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.473+0000 7f802d381640 1 -- 172.21.15.87:0/296909859 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80280b6500 con 0x7f80280b5bb0 2024-09-17T03:14:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.473+0000 7f8027fff640 1 --2- 172.21.15.87:0/296909859 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f80280b7b40 0x7f80280b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.474+0000 7f802d381640 1 --2- 172.21.15.87:0/296909859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80280b5bb0 0x7f80280b5fc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f80280cfce0 tx=0x7f8020006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.474+0000 7f80257fa640 1 -- 172.21.15.87:0/296909859 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8020015070 con 0x7f80280b5bb0 2024-09-17T03:14:14.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.474+0000 7f80257fa640 1 -- 172.21.15.87:0/296909859 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f802000fa40 con 0x7f80280b5bb0 2024-09-17T03:14:14.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.474+0000 7f802ede4640 1 -- 172.21.15.87:0/296909859 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80281538f0 con 0x7f80280b5bb0 2024-09-17T03:14:14.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.474+0000 7f80257fa640 1 -- 172.21.15.87:0/296909859 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802000fe90 con 0x7f80280b5bb0 2024-09-17T03:14:14.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.474+0000 7f802ede4640 1 -- 172.21.15.87:0/296909859 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8028153610 con 0x7f80280b5bb0 2024-09-17T03:14:14.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.475+0000 7f802ede4640 1 -- 172.21.15.87:0/296909859 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8028071f40 con 0x7f80280b5bb0 2024-09-17T03:14:14.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.476+0000 7f80257fa640 1 -- 172.21.15.87:0/296909859 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8020011430 con 0x7f80280b5bb0 2024-09-17T03:14:14.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.476+0000 7f80257fa640 1 --2- 172.21.15.87:0/296909859 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f800c078d30 0x7f800c07b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:14.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.476+0000 7f80257fa640 1 -- 172.21.15.87:0/296909859 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f802009b910 con 0x7f80280b5bb0 2024-09-17T03:14:14.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.476+0000 7f802cb80640 1 --2- 172.21.15.87:0/296909859 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f800c078d30 0x7f800c07b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:14.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.477+0000 7f802cb80640 1 --2- 172.21.15.87:0/296909859 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f800c078d30 0x7f800c07b240 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f801c009bb0 tx=0x7f801c005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:14.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.481+0000 7f80257fa640 1 -- 172.21.15.87:0/296909859 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8028071f40 con 0x7f80280b5bb0 2024-09-17T03:14:14.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.655+0000 7f802ede4640 1 -- 172.21.15.87:0/296909859 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f80280763c0 con 0x7f80280b5bb0 2024-09-17T03:14:14.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.656+0000 7f80257fa640 1 -- 172.21.15.87:0/296909859 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16706 (secure 0 0 0) 0x7f80280763c0 con 0x7f80280b5bb0 2024-09-17T03:14:14.655 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:14.655 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":22,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:14:13.672660+0000","last_up_change":"2024-09-17T03:14:01.833605+0000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:14:03.433404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:14:11.525547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T03:14:12.626396+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6825","nonce":2430409749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6827","nonce":2430409749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6831","nonce":2430409749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6829","nonce":2430409749}]},"public_addr":"172.21.15.87:6825/2430409749","cluster_addr":"172.21.15.87:6827/2430409749","heartbeat_back_addr":"172.21.15.87:6831/2430409749","heartbeat_front_addr":"172.21.15.87:6829/2430409749","state":["exists","up"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6809","nonce":3467605435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6811","nonce":3467605435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6815","nonce":3467605435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6813","nonce":3467605435}]},"public_addr":"172.21.15.87:6809/3467605435","cluster_addr":"172.21.15.87:6811/3467605435","heartbeat_back_addr":"172.21.15.87:6815/3467605435","heartbeat_front_addr":"172.21.15.87:6813/3467605435","state":["exists","up"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6801","nonce":1450834766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6803","nonce":1450834766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6807","nonce":1450834766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6805","nonce":1450834766}]},"public_addr":"172.21.15.87:6801/1450834766","cluster_addr":"172.21.15.87:6803/1450834766","heartbeat_back_addr":"172.21.15.87:6807/1450834766","heartbeat_front_addr":"172.21.15.87:6805/1450834766","state":["exists","up"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6817","nonce":4253161216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6819","nonce":4253161216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6823","nonce":4253161216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6821","nonce":4253161216}]},"public_addr":"172.21.15.87:6817/4253161216","cluster_addr":"172.21.15.87:6819/4253161216","heartbeat_back_addr":"172.21.15.87:6823/4253161216","heartbeat_front_addr":"172.21.15.87:6821/4253161216","state":["exists","up"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6803","nonce":2414907089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6805","nonce":2414907089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6809","nonce":2414907089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6807","nonce":2414907089}]},"public_addr":"172.21.15.149:6803/2414907089","cluster_addr":"172.21.15.149:6805/2414907089","heartbeat_back_addr":"172.21.15.149:6809/2414907089","heartbeat_front_addr":"172.21.15.149:6807/2414907089","state":["exists","up"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6827","nonce":2182999530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6829","nonce":2182999530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6833","nonce":2182999530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6831","nonce":2182999530}]},"public_addr":"172.21.15.149:6827/2182999530","cluster_addr":"172.21.15.149:6829/2182999530","heartbeat_back_addr":"172.21.15.149:6833/2182999530","heartbeat_front_addr":"172.21.15.149:6831/2182999530","state":["exists","up"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6811","nonce":4289519146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6813","nonce":4289519146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6817","nonce":4289519146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6815","nonce":4289519146}]},"public_addr":"172.21.15.149:6811/4289519146","cluster_addr":"172.21.15.149:6813/4289519146","heartbeat_back_addr":"172.21.15.149:6817/4289519146","heartbeat_front_addr":"172.21.15.149:6815/4289519146","state":["exists","up"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6819","nonce":2771010477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6821","nonce":2771010477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6825","nonce":2771010477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6823","nonce":2771010477}]},"public_addr":"172.21.15.149:6819/2771010477","cluster_addr":"172.21.15.149:6821/2771010477","heartbeat_back_addr":"172.21.15.149:6825/2771010477","heartbeat_front_addr":"172.21.15.149:6823/2771010477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.313297+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.642861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.654781+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.302606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.385791+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.915266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.121365+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.394432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:14:14.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.658+0000 7f802ede4640 1 -- 172.21.15.87:0/296909859 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f800c078d30 msgr2=0x7f800c07b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:14.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.658+0000 7f802ede4640 1 --2- 172.21.15.87:0/296909859 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f800c078d30 0x7f800c07b240 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f801c009bb0 tx=0x7f801c005f90 comp rx=0 tx=0).stop 2024-09-17T03:14:14.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.658+0000 7f802ede4640 1 -- 172.21.15.87:0/296909859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80280b5bb0 msgr2=0x7f80280b5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:14.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.658+0000 7f802ede4640 1 --2- 172.21.15.87:0/296909859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80280b5bb0 0x7f80280b5fc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f80280cfce0 tx=0x7f8020006660 comp rx=0 tx=0).stop 2024-09-17T03:14:14.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.658+0000 7f802ede4640 1 -- 172.21.15.87:0/296909859 shutdown_connections 2024-09-17T03:14:14.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.658+0000 7f802ede4640 1 --2- 172.21.15.87:0/296909859 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f800c078d30 0x7f800c07b240 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:14.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.658+0000 7f802ede4640 1 --2- 172.21.15.87:0/296909859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80280b5bb0 0x7f80280b5fc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:14.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.658+0000 7f802ede4640 1 --2- 172.21.15.87:0/296909859 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f80280b7b40 0x7f80280b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:14.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.658+0000 7f802ede4640 1 --2- 172.21.15.87:0/296909859 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f80180041c0 0x7f80280ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:14.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.658+0000 7f802ede4640 1 -- 172.21.15.87:0/296909859 >> 172.21.15.87:0/296909859 conn(0x7f8028069360 msgr2=0x7f802815ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:14.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.659+0000 7f802ede4640 1 -- 172.21.15.87:0/296909859 shutdown_connections 2024-09-17T03:14:14.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.659+0000 7f802ede4640 1 -- 172.21.15.87:0/296909859 wait complete. 2024-09-17T03:14:14.669 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:14:14.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.782+0000 7f2cd961d640 1 -- 172.21.15.87:0/838661181 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cc0005320 msgr2=0x7f2cc000b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:14.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.782+0000 7f2cd961d640 1 --2- 172.21.15.87:0/838661181 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cc0005320 0x7f2cc000b9e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00e3d0 tx=0x7f2cc0005830 comp rx=0 tx=0).stop 2024-09-17T03:14:14.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.782+0000 7f2cd961d640 1 -- 172.21.15.87:0/838661181 shutdown_connections 2024-09-17T03:14:14.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.782+0000 7f2cd961d640 1 --2- 172.21.15.87:0/838661181 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cc0005320 0x7f2cc000b9e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:14.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.782+0000 7f2cd961d640 1 -- 172.21.15.87:0/838661181 >> 172.21.15.87:0/838661181 conn(0x7f2cd408dae0 msgr2=0x7f2cd409d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:14.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.782+0000 7f2cd961d640 1 -- 172.21.15.87:0/838661181 shutdown_connections 2024-09-17T03:14:14.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.782+0000 7f2cd961d640 1 -- 172.21.15.87:0/838661181 wait complete. 2024-09-17T03:14:14.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.783+0000 7f2cd961d640 1 Processor -- start 2024-09-17T03:14:14.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.784+0000 7f2cd961d640 1 -- start start 2024-09-17T03:14:14.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.785+0000 7f2cd961d640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2cc0005320 0x7f2cd40ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:14.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.785+0000 7f2cd961d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cd4159f20 0x7f2cd415a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:14.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.785+0000 7f2cd961d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cd41f8a80 0x7f2cd41f8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:14.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.785+0000 7f2cd961d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd40d42d0 con 0x7f2cd4159f20 2024-09-17T03:14:14.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.785+0000 7f2cd961d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2cd40d4150 con 0x7f2cd41f8a80 2024-09-17T03:14:14.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.785+0000 7f2cd961d640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f2cd40d4450 con 0x7f2cc0005320 2024-09-17T03:14:14.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.785+0000 7f2cca7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cd4159f20 0x7f2cd415a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:14.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.785+0000 7f2cca7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cd4159f20 0x7f2cd415a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50304/0 (socket says 172.21.15.87:50304) 2024-09-17T03:14:14.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.785+0000 7f2cca7fc640 1 -- 172.21.15.87:0/3420372002 learned_addr learned my addr 172.21.15.87:0/3420372002 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:14.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.785+0000 7f2cd37fe640 1 --2- 172.21.15.87:0/3420372002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cd41f8a80 0x7f2cd41f8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:14.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.785+0000 7f2cd2ffd640 1 --2- 172.21.15.87:0/3420372002 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2cc0005320 0x7f2cd40ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:14.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.785+0000 7f2cca7fc640 1 -- 172.21.15.87:0/3420372002 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2cc0005320 msgr2=0x7f2cd40ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:14.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.785+0000 7f2cca7fc640 1 --2- 172.21.15.87:0/3420372002 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2cc0005320 0x7f2cd40ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:14.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.785+0000 7f2cca7fc640 1 -- 172.21.15.87:0/3420372002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cd41f8a80 msgr2=0x7f2cd41f8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:14.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.785+0000 7f2cca7fc640 1 --2- 172.21.15.87:0/3420372002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cd41f8a80 0x7f2cd41f8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:14.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.785+0000 7f2cca7fc640 1 -- 172.21.15.87:0/3420372002 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cd41f93d0 con 0x7f2cd4159f20 2024-09-17T03:14:14.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.785+0000 7f2cca7fc640 1 --2- 172.21.15.87:0/3420372002 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cd4159f20 0x7f2cd415a330 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc006480 tx=0x7f2cbc0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:14.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.785+0000 7f2ccbfff640 1 -- 172.21.15.87:0/3420372002 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc015070 con 0x7f2cd4159f20 2024-09-17T03:14:14.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.786+0000 7f2ccbfff640 1 -- 172.21.15.87:0/3420372002 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cbc011860 con 0x7f2cd4159f20 2024-09-17T03:14:14.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.786+0000 7f2cd961d640 1 -- 172.21.15.87:0/3420372002 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cd415ac50 con 0x7f2cd4159f20 2024-09-17T03:14:14.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.786+0000 7f2ccbfff640 1 -- 172.21.15.87:0/3420372002 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc011cb0 con 0x7f2cd4159f20 2024-09-17T03:14:14.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.786+0000 7f2cd961d640 1 -- 172.21.15.87:0/3420372002 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cd415b210 con 0x7f2cd4159f20 2024-09-17T03:14:14.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.787+0000 7f2cd961d640 1 -- 172.21.15.87:0/3420372002 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cd40d0eb0 con 0x7f2cd4159f20 2024-09-17T03:14:14.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.787+0000 7f2ccbfff640 1 -- 172.21.15.87:0/3420372002 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2cbc018070 con 0x7f2cd4159f20 2024-09-17T03:14:14.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.788+0000 7f2ccbfff640 1 --2- 172.21.15.87:0/3420372002 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f2cb4078bc0 0x7f2cb407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:14.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.788+0000 7f2ccbfff640 1 -- 172.21.15.87:0/3420372002 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f2cbc09b920 con 0x7f2cd4159f20 2024-09-17T03:14:14.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.788+0000 7f2cd2ffd640 1 --2- 172.21.15.87:0/3420372002 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f2cb4078bc0 0x7f2cb407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:14.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.789+0000 7f2cd2ffd640 1 --2- 172.21.15.87:0/3420372002 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f2cb4078bc0 0x7f2cb407b0d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4006460 tx=0x7f2cc4006b10 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:14.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.792+0000 7f2ccbfff640 1 -- 172.21.15.87:0/3420372002 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f2cd41f93d0 con 0x7f2cd4159f20 2024-09-17T03:14:14.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.966+0000 7f2cd961d640 1 -- 172.21.15.87:0/3420372002 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2cd415a9e0 con 0x7f2cd4159f20 2024-09-17T03:14:14.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.967+0000 7f2ccbfff640 1 -- 172.21.15.87:0/3420372002 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16706 (secure 0 0 0) 0x7f2cbc063d70 con 0x7f2cd4159f20 2024-09-17T03:14:14.966 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:14.966 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":22,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:14:13.672660+0000","last_up_change":"2024-09-17T03:14:01.833605+0000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:14:03.433404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:14:11.525547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T03:14:12.626396+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6825","nonce":2430409749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6827","nonce":2430409749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6831","nonce":2430409749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6829","nonce":2430409749}]},"public_addr":"172.21.15.87:6825/2430409749","cluster_addr":"172.21.15.87:6827/2430409749","heartbeat_back_addr":"172.21.15.87:6831/2430409749","heartbeat_front_addr":"172.21.15.87:6829/2430409749","state":["exists","up"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6809","nonce":3467605435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6811","nonce":3467605435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6815","nonce":3467605435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6813","nonce":3467605435}]},"public_addr":"172.21.15.87:6809/3467605435","cluster_addr":"172.21.15.87:6811/3467605435","heartbeat_back_addr":"172.21.15.87:6815/3467605435","heartbeat_front_addr":"172.21.15.87:6813/3467605435","state":["exists","up"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6801","nonce":1450834766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6803","nonce":1450834766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6807","nonce":1450834766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6805","nonce":1450834766}]},"public_addr":"172.21.15.87:6801/1450834766","cluster_addr":"172.21.15.87:6803/1450834766","heartbeat_back_addr":"172.21.15.87:6807/1450834766","heartbeat_front_addr":"172.21.15.87:6805/1450834766","state":["exists","up"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6817","nonce":4253161216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6819","nonce":4253161216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6823","nonce":4253161216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6821","nonce":4253161216}]},"public_addr":"172.21.15.87:6817/4253161216","cluster_addr":"172.21.15.87:6819/4253161216","heartbeat_back_addr":"172.21.15.87:6823/4253161216","heartbeat_front_addr":"172.21.15.87:6821/4253161216","state":["exists","up"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6803","nonce":2414907089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6805","nonce":2414907089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6809","nonce":2414907089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6807","nonce":2414907089}]},"public_addr":"172.21.15.149:6803/2414907089","cluster_addr":"172.21.15.149:6805/2414907089","heartbeat_back_addr":"172.21.15.149:6809/2414907089","heartbeat_front_addr":"172.21.15.149:6807/2414907089","state":["exists","up"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6827","nonce":2182999530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6829","nonce":2182999530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6833","nonce":2182999530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6831","nonce":2182999530}]},"public_addr":"172.21.15.149:6827/2182999530","cluster_addr":"172.21.15.149:6829/2182999530","heartbeat_back_addr":"172.21.15.149:6833/2182999530","heartbeat_front_addr":"172.21.15.149:6831/2182999530","state":["exists","up"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6811","nonce":4289519146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6813","nonce":4289519146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6817","nonce":4289519146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6815","nonce":4289519146}]},"public_addr":"172.21.15.149:6811/4289519146","cluster_addr":"172.21.15.149:6813/4289519146","heartbeat_back_addr":"172.21.15.149:6817/4289519146","heartbeat_front_addr":"172.21.15.149:6815/4289519146","state":["exists","up"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6819","nonce":2771010477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6821","nonce":2771010477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6825","nonce":2771010477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6823","nonce":2771010477}]},"public_addr":"172.21.15.149:6819/2771010477","cluster_addr":"172.21.15.149:6821/2771010477","heartbeat_back_addr":"172.21.15.149:6825/2771010477","heartbeat_front_addr":"172.21.15.149:6823/2771010477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.313297+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.642861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.654781+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.302606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.385791+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.915266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.121365+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.394432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:14:14.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.969+0000 7f2cd961d640 1 -- 172.21.15.87:0/3420372002 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f2cb4078bc0 msgr2=0x7f2cb407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:14.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.969+0000 7f2cd961d640 1 --2- 172.21.15.87:0/3420372002 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f2cb4078bc0 0x7f2cb407b0d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4006460 tx=0x7f2cc4006b10 comp rx=0 tx=0).stop 2024-09-17T03:14:14.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.969+0000 7f2cd961d640 1 -- 172.21.15.87:0/3420372002 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cd4159f20 msgr2=0x7f2cd415a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:14.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.969+0000 7f2cd961d640 1 --2- 172.21.15.87:0/3420372002 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cd4159f20 0x7f2cd415a330 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc006480 tx=0x7f2cbc0039c0 comp rx=0 tx=0).stop 2024-09-17T03:14:14.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.969+0000 7f2cd961d640 1 -- 172.21.15.87:0/3420372002 shutdown_connections 2024-09-17T03:14:14.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.969+0000 7f2cd961d640 1 --2- 172.21.15.87:0/3420372002 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f2cb4078bc0 0x7f2cb407b0d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:14.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.969+0000 7f2cd961d640 1 --2- 172.21.15.87:0/3420372002 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cd41f8a80 0x7f2cd41f8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:14.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.969+0000 7f2cd961d640 1 --2- 172.21.15.87:0/3420372002 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cd4159f20 0x7f2cd415a330 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:14.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.969+0000 7f2cd961d640 1 --2- 172.21.15.87:0/3420372002 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2cc0005320 0x7f2cd40ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:14.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.969+0000 7f2cd961d640 1 -- 172.21.15.87:0/3420372002 >> 172.21.15.87:0/3420372002 conn(0x7f2cd408dae0 msgr2=0x7f2cd40b7520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:14.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.970+0000 7f2cd961d640 1 -- 172.21.15.87:0/3420372002 shutdown_connections 2024-09-17T03:14:14.969 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:14.970+0000 7f2cd961d640 1 -- 172.21.15.87:0/3420372002 wait complete. 2024-09-17T03:14:14.980 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T03:14:15.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.096+0000 7fd7ce84d640 1 -- 172.21.15.87:0/633762075 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7b00051c0 msgr2=0x7fd7b000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:15.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.096+0000 7fd7ce84d640 1 --2- 172.21.15.87:0/633762075 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7b00051c0 0x7fd7b000b9c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd7c80b3ba0 tx=0x7fd7b0005830 comp rx=0 tx=0).stop 2024-09-17T03:14:15.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.096+0000 7fd7ccdea640 1 -- 172.21.15.87:0/633762075 reap_dead start 2024-09-17T03:14:15.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.096+0000 7fd7ce84d640 1 -- 172.21.15.87:0/633762075 shutdown_connections 2024-09-17T03:14:15.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.096+0000 7fd7ce84d640 1 -- 172.21.15.87:0/633762075 >> 172.21.15.87:0/633762075 conn(0x7fd7c809d840 msgr2=0x7fd7c808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:15.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.096+0000 7fd7ce84d640 1 -- 172.21.15.87:0/633762075 shutdown_connections 2024-09-17T03:14:15.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.096+0000 7fd7ce84d640 1 -- 172.21.15.87:0/633762075 wait complete. 2024-09-17T03:14:15.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.097+0000 7fd7ce84d640 1 Processor -- start 2024-09-17T03:14:15.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.098+0000 7fd7ce84d640 1 -- start start 2024-09-17T03:14:15.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.099+0000 7fd7ce84d640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7b00051c0 0x7fd7c80ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:15.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.099+0000 7fd7ce84d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7c81f5d80 0x7fd7c815de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:15.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.099+0000 7fd7ce84d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7c815e3b0 0x7fd7c814dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:15.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.099+0000 7fd7ce84d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c80b3660 con 0x7fd7c81f5d80 2024-09-17T03:14:15.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.099+0000 7fd7ce84d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd7c806e340 con 0x7fd7c815e3b0 2024-09-17T03:14:15.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.099+0000 7fd7ce84d640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd7c80b37e0 con 0x7fd7b00051c0 2024-09-17T03:14:15.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.099+0000 7fd7c77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7c81f5d80 0x7fd7c815de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:15.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.099+0000 7fd7c77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7c81f5d80 0x7fd7c815de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50330/0 (socket says 172.21.15.87:50330) 2024-09-17T03:14:15.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.099+0000 7fd7c77fe640 1 -- 172.21.15.87:0/733872226 learned_addr learned my addr 172.21.15.87:0/733872226 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:15.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.099+0000 7fd7c7fff640 1 --2- 172.21.15.87:0/733872226 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7b00051c0 0x7fd7c80ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:15.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.099+0000 7fd7ccdea640 1 --2- 172.21.15.87:0/733872226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7c815e3b0 0x7fd7c814dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:15.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.099+0000 7fd7c77fe640 1 -- 172.21.15.87:0/733872226 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7b00051c0 msgr2=0x7fd7c80ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:15.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.099+0000 7fd7c77fe640 1 --2- 172.21.15.87:0/733872226 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7b00051c0 0x7fd7c80ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:15.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.099+0000 7fd7c77fe640 1 -- 172.21.15.87:0/733872226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7c815e3b0 msgr2=0x7fd7c814dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:15.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.099+0000 7fd7c77fe640 1 --2- 172.21.15.87:0/733872226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7c815e3b0 0x7fd7c814dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:15.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.099+0000 7fd7c77fe640 1 -- 172.21.15.87:0/733872226 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7c80cd430 con 0x7fd7c81f5d80 2024-09-17T03:14:15.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.099+0000 7fd7c77fe640 1 --2- 172.21.15.87:0/733872226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7c81f5d80 0x7fd7c815de70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc006680 tx=0x7fd7bc0066b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:15.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.100+0000 7fd7c4ff9640 1 -- 172.21.15.87:0/733872226 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc015070 con 0x7fd7c81f5d80 2024-09-17T03:14:15.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.100+0000 7fd7c4ff9640 1 -- 172.21.15.87:0/733872226 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7bc011920 con 0x7fd7c81f5d80 2024-09-17T03:14:15.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.100+0000 7fd7ce84d640 1 -- 172.21.15.87:0/733872226 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7c815e7c0 con 0x7fd7c81f5d80 2024-09-17T03:14:15.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.100+0000 7fd7c4ff9640 1 -- 172.21.15.87:0/733872226 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7bc011d70 con 0x7fd7c81f5d80 2024-09-17T03:14:15.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.100+0000 7fd7ce84d640 1 -- 172.21.15.87:0/733872226 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7c80cd980 con 0x7fd7c81f5d80 2024-09-17T03:14:15.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.101+0000 7fd7ce84d640 1 -- 172.21.15.87:0/733872226 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7c80cba30 con 0x7fd7c81f5d80 2024-09-17T03:14:15.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.101+0000 7fd7c4ff9640 1 -- 172.21.15.87:0/733872226 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd7bc018070 con 0x7fd7c81f5d80 2024-09-17T03:14:15.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.102+0000 7fd7c4ff9640 1 --2- 172.21.15.87:0/733872226 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd7b4078bc0 0x7fd7b407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:15.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.102+0000 7fd7c4ff9640 1 -- 172.21.15.87:0/733872226 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fd7bc09baa0 con 0x7fd7c81f5d80 2024-09-17T03:14:15.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.102+0000 7fd7c7fff640 1 --2- 172.21.15.87:0/733872226 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd7b4078bc0 0x7fd7b407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:15.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.103+0000 7fd7c7fff640 1 --2- 172.21.15.87:0/733872226 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd7b4078bc0 0x7fd7b407b0d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8000c00 tx=0x7fd7b8006b70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:15.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.107+0000 7fd7c4ff9640 1 -- 172.21.15.87:0/733872226 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd7bc063920 con 0x7fd7c81f5d80 2024-09-17T03:14:15.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.315+0000 7fd7ce84d640 1 -- 172.21.15.87:0/733872226 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd7c80cde70 con 0x7fd7c81f5d80 2024-09-17T03:14:15.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.324+0000 7fd7c4ff9640 1 -- 172.21.15.87:0/733872226 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) ==== 118+0+0 (secure 0 0 0) 0x7fd7bc068350 con 0x7fd7c81f5d80 2024-09-17T03:14:15.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.326+0000 7fd7ce84d640 1 -- 172.21.15.87:0/733872226 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd7b4078bc0 msgr2=0x7fd7b407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:15.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.326+0000 7fd7ce84d640 1 --2- 172.21.15.87:0/733872226 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd7b4078bc0 0x7fd7b407b0d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8000c00 tx=0x7fd7b8006b70 comp rx=0 tx=0).stop 2024-09-17T03:14:15.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.326+0000 7fd7ce84d640 1 -- 172.21.15.87:0/733872226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7c81f5d80 msgr2=0x7fd7c815de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:15.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.326+0000 7fd7ce84d640 1 --2- 172.21.15.87:0/733872226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7c81f5d80 0x7fd7c815de70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc006680 tx=0x7fd7bc0066b0 comp rx=0 tx=0).stop 2024-09-17T03:14:15.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.326+0000 7fd7ce84d640 1 -- 172.21.15.87:0/733872226 shutdown_connections 2024-09-17T03:14:15.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.326+0000 7fd7ce84d640 1 --2- 172.21.15.87:0/733872226 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd7b4078bc0 0x7fd7b407b0d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:15.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.326+0000 7fd7ce84d640 1 --2- 172.21.15.87:0/733872226 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd7c815e3b0 0x7fd7c814dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:15.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.326+0000 7fd7ce84d640 1 --2- 172.21.15.87:0/733872226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7c81f5d80 0x7fd7c815de70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:15.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.326+0000 7fd7ce84d640 1 --2- 172.21.15.87:0/733872226 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd7b00051c0 0x7fd7c80ce180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:15.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.326+0000 7fd7ce84d640 1 -- 172.21.15.87:0/733872226 >> 172.21.15.87:0/733872226 conn(0x7fd7c809d840 msgr2=0x7fd7c806ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:15.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.326+0000 7fd7ce84d640 1 -- 172.21.15.87:0/733872226 shutdown_connections 2024-09-17T03:14:15.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.326+0000 7fd7ce84d640 1 -- 172.21.15.87:0/733872226 wait complete. 2024-09-17T03:14:15.337 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T03:14:15.338 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T03:14:15.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.454+0000 7fdb41f8b640 1 -- 172.21.15.87:0/3575095071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb24003fd0 msgr2=0x7fdb24004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:15.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.454+0000 7fdb41f8b640 1 --2- 172.21.15.87:0/3575095071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb24003fd0 0x7fdb24004800 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009b20 tx=0x7fdb24005830 comp rx=0 tx=0).stop 2024-09-17T03:14:15.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.454+0000 7fdb3bfff640 1 -- 172.21.15.87:0/3575095071 reap_dead start 2024-09-17T03:14:15.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.454+0000 7fdb41f8b640 1 -- 172.21.15.87:0/3575095071 shutdown_connections 2024-09-17T03:14:15.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.454+0000 7fdb41f8b640 1 -- 172.21.15.87:0/3575095071 >> 172.21.15.87:0/3575095071 conn(0x7fdb3c08bfc0 msgr2=0x7fdb3c09d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:15.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.454+0000 7fdb41f8b640 1 -- 172.21.15.87:0/3575095071 shutdown_connections 2024-09-17T03:14:15.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.454+0000 7fdb41f8b640 1 -- 172.21.15.87:0/3575095071 wait complete. 2024-09-17T03:14:15.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.456+0000 7fdb41f8b640 1 Processor -- start 2024-09-17T03:14:15.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.456+0000 7fdb41f8b640 1 -- start start 2024-09-17T03:14:15.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.457+0000 7fdb41f8b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb24003fd0 0x7fdb3c15f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:15.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.458+0000 7fdb41f8b640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdb3c150a10 0x7fdb3c150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:15.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.458+0000 7fdb41f8b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb3c0a14f0 0x7fdb3c0a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:15.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.458+0000 7fdb41f8b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c0cf840 con 0x7fdb24003fd0 2024-09-17T03:14:15.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.458+0000 7fdb41f8b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdb3c0cf6c0 con 0x7fdb3c0a14f0 2024-09-17T03:14:15.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.458+0000 7fdb41f8b640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fdb3c0cf9c0 con 0x7fdb3c150a10 2024-09-17T03:14:15.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.458+0000 7fdb3b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb24003fd0 0x7fdb3c15f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:15.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.458+0000 7fdb3b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb24003fd0 0x7fdb3c15f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39860/0 (socket says 172.21.15.87:39860) 2024-09-17T03:14:15.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.458+0000 7fdb3b7fe640 1 -- 172.21.15.87:0/3675617847 learned_addr learned my addr 172.21.15.87:0/3675617847 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:15.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.458+0000 7fdb3affd640 1 --2- 172.21.15.87:0/3675617847 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdb3c150a10 0x7fdb3c150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:15.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.458+0000 7fdb3bfff640 1 --2- 172.21.15.87:0/3675617847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb3c0a14f0 0x7fdb3c0a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:15.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.458+0000 7fdb3b7fe640 1 -- 172.21.15.87:0/3675617847 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdb3c150a10 msgr2=0x7fdb3c150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:15.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.458+0000 7fdb3b7fe640 1 --2- 172.21.15.87:0/3675617847 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdb3c150a10 0x7fdb3c150e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:15.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.458+0000 7fdb3b7fe640 1 -- 172.21.15.87:0/3675617847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb3c0a14f0 msgr2=0x7fdb3c0a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:15.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.458+0000 7fdb3b7fe640 1 --2- 172.21.15.87:0/3675617847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb3c0a14f0 0x7fdb3c0a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:15.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.458+0000 7fdb3b7fe640 1 -- 172.21.15.87:0/3675617847 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb3c0b83e0 con 0x7fdb24003fd0 2024-09-17T03:14:15.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.458+0000 7fdb3bfff640 1 --2- 172.21.15.87:0/3675617847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb3c0a14f0 0x7fdb3c0a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:14:15.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.458+0000 7fdb3b7fe640 1 --2- 172.21.15.87:0/3675617847 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb24003fd0 0x7fdb3c15f470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c011ae0 tx=0x7fdb2c009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:15.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.458+0000 7fdb23fff640 1 -- 172.21.15.87:0/3675617847 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c01e070 con 0x7fdb24003fd0 2024-09-17T03:14:15.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.458+0000 7fdb23fff640 1 -- 172.21.15.87:0/3675617847 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb2c00fc10 con 0x7fdb24003fd0 2024-09-17T03:14:15.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.458+0000 7fdb23fff640 1 -- 172.21.15.87:0/3675617847 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c019410 con 0x7fdb24003fd0 2024-09-17T03:14:15.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.458+0000 7fdb41f8b640 1 -- 172.21.15.87:0/3675617847 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb3c0b86c0 con 0x7fdb24003fd0 2024-09-17T03:14:15.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.458+0000 7fdb41f8b640 1 -- 172.21.15.87:0/3675617847 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb3c0b8c80 con 0x7fdb24003fd0 2024-09-17T03:14:15.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.459+0000 7fdb41f8b640 1 -- 172.21.15.87:0/3675617847 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb3c0d43e0 con 0x7fdb24003fd0 2024-09-17T03:14:15.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.460+0000 7fdb23fff640 1 -- 172.21.15.87:0/3675617847 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdb2c021070 con 0x7fdb24003fd0 2024-09-17T03:14:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.460+0000 7fdb23fff640 1 --2- 172.21.15.87:0/3675617847 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdb28078b90 0x7fdb2807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.460+0000 7fdb23fff640 1 -- 172.21.15.87:0/3675617847 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fdb2c09b4a0 con 0x7fdb24003fd0 2024-09-17T03:14:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.461+0000 7fdb3affd640 1 --2- 172.21.15.87:0/3675617847 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdb28078b90 0x7fdb2807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:15.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.461+0000 7fdb3affd640 1 --2- 172.21.15.87:0/3675617847 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdb28078b90 0x7fdb2807b0a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdb30035000 tx=0x7fdb300046c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:15.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.465+0000 7fdb23fff640 1 -- 172.21.15.87:0/3675617847 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fdb2c063290 con 0x7fdb24003fd0 2024-09-17T03:14:15.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:15.674+0000 7fdb41f8b640 1 -- 172.21.15.87:0/3675617847 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fdb3c0d0c90 con 0x7fdb24003fd0 2024-09-17T03:14:16.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.334+0000 7fdb23fff640 1 -- 172.21.15.87:0/3675617847 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) ==== 122+0+0 (secure 0 0 0) 0x7fdb2c067cc0 con 0x7fdb24003fd0 2024-09-17T03:14:16.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.337+0000 7fdb41f8b640 1 -- 172.21.15.87:0/3675617847 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdb28078b90 msgr2=0x7fdb2807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:16.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.337+0000 7fdb41f8b640 1 --2- 172.21.15.87:0/3675617847 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdb28078b90 0x7fdb2807b0a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdb30035000 tx=0x7fdb300046c0 comp rx=0 tx=0).stop 2024-09-17T03:14:16.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.337+0000 7fdb41f8b640 1 -- 172.21.15.87:0/3675617847 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb24003fd0 msgr2=0x7fdb3c15f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:16.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.337+0000 7fdb41f8b640 1 --2- 172.21.15.87:0/3675617847 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb24003fd0 0x7fdb3c15f470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c011ae0 tx=0x7fdb2c009e60 comp rx=0 tx=0).stop 2024-09-17T03:14:16.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.337+0000 7fdb41f8b640 1 -- 172.21.15.87:0/3675617847 shutdown_connections 2024-09-17T03:14:16.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.338+0000 7fdb41f8b640 1 --2- 172.21.15.87:0/3675617847 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdb28078b90 0x7fdb2807b0a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:16.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.338+0000 7fdb41f8b640 1 --2- 172.21.15.87:0/3675617847 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb3c0a14f0 0x7fdb3c0a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:16.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.338+0000 7fdb41f8b640 1 --2- 172.21.15.87:0/3675617847 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdb3c150a10 0x7fdb3c150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:16.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.338+0000 7fdb41f8b640 1 --2- 172.21.15.87:0/3675617847 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb24003fd0 0x7fdb3c15f470 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:16.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.338+0000 7fdb41f8b640 1 -- 172.21.15.87:0/3675617847 >> 172.21.15.87:0/3675617847 conn(0x7fdb3c08bfc0 msgr2=0x7fdb3c0b5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:16.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.338+0000 7fdb41f8b640 1 -- 172.21.15.87:0/3675617847 shutdown_connections 2024-09-17T03:14:16.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.338+0000 7fdb41f8b640 1 -- 172.21.15.87:0/3675617847 wait complete. 2024-09-17T03:14:16.350 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T03:14:16.350 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:14:16.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.470+0000 7f95a6120640 1 -- 172.21.15.87:0/1716244134 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95a00cefb0 msgr2=0x7f95a00cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:16.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.470+0000 7f95a6120640 1 --2- 172.21.15.87:0/1716244134 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95a00cefb0 0x7f95a00cf3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f95a008c800 tx=0x7f959802a820 comp rx=0 tx=0).stop 2024-09-17T03:14:16.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.470+0000 7f95a6120640 1 -- 172.21.15.87:0/1716244134 shutdown_connections 2024-09-17T03:14:16.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.470+0000 7f95a6120640 1 --2- 172.21.15.87:0/1716244134 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95a00cefb0 0x7f95a00cf3c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:16.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.470+0000 7f95a6120640 1 --2- 172.21.15.87:0/1716244134 >> v2:172.21.15.149:3300/0 conn(0x7f95a00d2c50 0x7f95a00d3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:16.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.470+0000 7f95a6120640 1 -- 172.21.15.87:0/1716244134 >> 172.21.15.87:0/1716244134 conn(0x7f95a008dae0 msgr2=0x7f95a009d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:16.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.470+0000 7f95a6120640 1 -- 172.21.15.87:0/1716244134 shutdown_connections 2024-09-17T03:14:16.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.470+0000 7f95a6120640 1 -- 172.21.15.87:0/1716244134 wait complete. 2024-09-17T03:14:16.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.472+0000 7f95a6120640 1 Processor -- start 2024-09-17T03:14:16.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.472+0000 7f95a6120640 1 -- start start 2024-09-17T03:14:16.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.473+0000 7f95a6120640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95a00cefb0 0x7f95a014a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:16.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.473+0000 7f95a6120640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95a00d2c50 0x7f95a015f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:16.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.473+0000 7f95a6120640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95a00d4150 0x7f95a01f8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:16.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.473+0000 7f95a6120640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f95a00cfa80 con 0x7f95a00d4150 2024-09-17T03:14:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.473+0000 7f95a6120640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f95a00cf900 con 0x7f95a00d2c50 2024-09-17T03:14:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.473+0000 7f95a6120640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f95a00cfc00 con 0x7f95a00cefb0 2024-09-17T03:14:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.473+0000 7f959ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95a00d4150 0x7f95a01f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.473+0000 7f959ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95a00d4150 0x7f95a01f8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39882/0 (socket says 172.21.15.87:39882) 2024-09-17T03:14:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.473+0000 7f959ffff640 1 -- 172.21.15.87:0/553903193 learned_addr learned my addr 172.21.15.87:0/553903193 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.474+0000 7f959ffff640 1 -- 172.21.15.87:0/553903193 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95a00cefb0 msgr2=0x7f95a014a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.474+0000 7f959effd640 1 --2- 172.21.15.87:0/553903193 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95a00d2c50 0x7f95a015f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.474+0000 7f959ffff640 1 --2- 172.21.15.87:0/553903193 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95a00cefb0 0x7f95a014a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.474+0000 7f959ffff640 1 -- 172.21.15.87:0/553903193 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95a00d2c50 msgr2=0x7f95a015f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.474+0000 7f959ffff640 1 --2- 172.21.15.87:0/553903193 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95a00d2c50 0x7f95a015f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.474+0000 7f959ffff640 1 -- 172.21.15.87:0/553903193 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a014ad90 con 0x7f95a00d4150 2024-09-17T03:14:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.474+0000 7f959ffff640 1 --2- 172.21.15.87:0/553903193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95a00d4150 0x7f95a01f8760 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9598009ca0 tx=0x7f9598032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:16.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.474+0000 7f9587fff640 1 -- 172.21.15.87:0/553903193 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598037040 con 0x7f95a00d4150 2024-09-17T03:14:16.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.474+0000 7f9587fff640 1 -- 172.21.15.87:0/553903193 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9598042070 con 0x7f95a00d4150 2024-09-17T03:14:16.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.474+0000 7f95a6120640 1 -- 172.21.15.87:0/553903193 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a015fca0 con 0x7f95a00d4150 2024-09-17T03:14:16.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.474+0000 7f9587fff640 1 -- 172.21.15.87:0/553903193 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95980042a0 con 0x7f95a00d4150 2024-09-17T03:14:16.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.474+0000 7f95a6120640 1 -- 172.21.15.87:0/553903193 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a01e8320 con 0x7f95a00d4150 2024-09-17T03:14:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.475+0000 7f95a6120640 1 -- 172.21.15.87:0/553903193 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95a00d00b0 con 0x7f95a00d4150 2024-09-17T03:14:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.476+0000 7f9587fff640 1 -- 172.21.15.87:0/553903193 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9598004440 con 0x7f95a00d4150 2024-09-17T03:14:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.476+0000 7f9587fff640 1 --2- 172.21.15.87:0/553903193 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f958c078b90 0x7f958c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.476+0000 7f9587fff640 1 -- 172.21.15.87:0/553903193 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f95980bbca0 con 0x7f95a00d4150 2024-09-17T03:14:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.476+0000 7f959f7fe640 1 --2- 172.21.15.87:0/553903193 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f958c078b90 0x7f958c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:16.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.477+0000 7f959f7fe640 1 --2- 172.21.15.87:0/553903193 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f958c078b90 0x7f958c07b0a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f959000a8f0 tx=0x7f9590005bd0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:16.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.481+0000 7f9587fff640 1 -- 172.21.15.87:0/553903193 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f95a014ad90 con 0x7f95a00d4150 2024-09-17T03:14:16.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.688+0000 7f95a6120640 1 -- 172.21.15.87:0/553903193 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95a015fa10 con 0x7f95a00d4150 2024-09-17T03:14:16.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.688+0000 7f9587fff640 1 -- 172.21.15.87:0/553903193 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5217 (secure 0 0 0) 0x7f95980840e0 con 0x7f95a00d4150 2024-09-17T03:14:16.687 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:16.687 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":6,"btime":"2024-09-17T03:14:16:330267+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4289,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/110106418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":110106418},{"type":"v1","addr":"172.21.15.149:6835","nonce":110106418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1336626873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1336626873},{"type":"v1","addr":"172.21.15.149:6837","nonce":1336626873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/666860906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":666860906},{"type":"v1","addr":"172.21.15.87:6833","nonce":666860906}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:14:13.672621+0000","modified":"2024-09-17T03:14:15.675568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4398":{"gid":4398,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/1831073338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1831073338},{"type":"v1","addr":"172.21.15.87:6835","nonce":1831073338}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4398,"qdb_cluster":[4398]},"id":1}]} 2024-09-17T03:14:16.688 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 6 2024-09-17T03:14:16.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.691+0000 7f95a6120640 1 -- 172.21.15.87:0/553903193 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f958c078b90 msgr2=0x7f958c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:16.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.691+0000 7f95a6120640 1 --2- 172.21.15.87:0/553903193 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f958c078b90 0x7f958c07b0a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f959000a8f0 tx=0x7f9590005bd0 comp rx=0 tx=0).stop 2024-09-17T03:14:16.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.691+0000 7f95a6120640 1 -- 172.21.15.87:0/553903193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95a00d4150 msgr2=0x7f95a01f8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:16.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.691+0000 7f95a6120640 1 --2- 172.21.15.87:0/553903193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95a00d4150 0x7f95a01f8760 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9598009ca0 tx=0x7f9598032d10 comp rx=0 tx=0).stop 2024-09-17T03:14:16.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.691+0000 7f95a6120640 1 -- 172.21.15.87:0/553903193 shutdown_connections 2024-09-17T03:14:16.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.691+0000 7f95a6120640 1 --2- 172.21.15.87:0/553903193 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f958c078b90 0x7f958c07b0a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:16.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.691+0000 7f95a6120640 1 --2- 172.21.15.87:0/553903193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95a00d4150 0x7f95a01f8760 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:16.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.691+0000 7f95a6120640 1 --2- 172.21.15.87:0/553903193 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f95a00d2c50 0x7f95a015f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:16.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.691+0000 7f95a6120640 1 --2- 172.21.15.87:0/553903193 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f95a00cefb0 0x7f95a014a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:16.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.691+0000 7f95a6120640 1 -- 172.21.15.87:0/553903193 >> 172.21.15.87:0/553903193 conn(0x7f95a008dae0 msgr2=0x7f95a015c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:16.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.691+0000 7f95a6120640 1 -- 172.21.15.87:0/553903193 shutdown_connections 2024-09-17T03:14:16.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.691+0000 7f95a6120640 1 -- 172.21.15.87:0/553903193 wait complete. 2024-09-17T03:14:16.701 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:14:16.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.813+0000 7f652b382640 1 -- 172.21.15.87:0/3998153925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f652415ecf0 msgr2=0x7f652415f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:16.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.813+0000 7f652b382640 1 --2- 172.21.15.87:0/3998153925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f652415ecf0 0x7f652415f100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f65240b3e80 tx=0x7f651402a530 comp rx=0 tx=0).stop 2024-09-17T03:14:16.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.813+0000 7f652b382640 1 -- 172.21.15.87:0/3998153925 shutdown_connections 2024-09-17T03:14:16.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.813+0000 7f652b382640 1 --2- 172.21.15.87:0/3998153925 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f65240b5ef0 0x7f65240b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:16.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.813+0000 7f652b382640 1 --2- 172.21.15.87:0/3998153925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f652415ecf0 0x7f652415f100 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:16.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.813+0000 7f652b382640 1 -- 172.21.15.87:0/3998153925 >> 172.21.15.87:0/3998153925 conn(0x7f65240d4470 msgr2=0x7f65240d4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:16.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.813+0000 7f652b382640 1 -- 172.21.15.87:0/3998153925 shutdown_connections 2024-09-17T03:14:16.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.813+0000 7f652b382640 1 -- 172.21.15.87:0/3998153925 wait complete. 2024-09-17T03:14:16.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.815+0000 7f652b382640 1 Processor -- start 2024-09-17T03:14:16.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.815+0000 7f652b382640 1 -- start start 2024-09-17T03:14:16.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.816+0000 7f652b382640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65240b5ef0 0x7f65240cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:16.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.816+0000 7f652b382640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f652415c2d0 0x7f65240caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:16.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.816+0000 7f652b382640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f652415ecf0 0x7f65241f6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:16.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.816+0000 7f652b382640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f65240d2db0 con 0x7f652415c2d0 2024-09-17T03:14:16.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.816+0000 7f652b382640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f65240d2c30 con 0x7f65240b5ef0 2024-09-17T03:14:16.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.816+0000 7f652b382640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f65240d2f30 con 0x7f652415ecf0 2024-09-17T03:14:16.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.816+0000 7f652891d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f652415c2d0 0x7f65240caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:16.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.816+0000 7f652891d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f652415c2d0 0x7f65240caa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39900/0 (socket says 172.21.15.87:39900) 2024-09-17T03:14:16.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.816+0000 7f652891d640 1 -- 172.21.15.87:0/2949146108 learned_addr learned my addr 172.21.15.87:0/2949146108 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:16.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.816+0000 7f652891d640 1 -- 172.21.15.87:0/2949146108 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f652415ecf0 msgr2=0x7f65241f6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:16.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.816+0000 7f652891d640 1 --2- 172.21.15.87:0/2949146108 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f652415ecf0 0x7f65241f6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:16.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.816+0000 7f652891d640 1 -- 172.21.15.87:0/2949146108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65240b5ef0 msgr2=0x7f65240cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:16.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.816+0000 7f652891d640 1 --2- 172.21.15.87:0/2949146108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65240b5ef0 0x7f65240cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:16.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.816+0000 7f652891d640 1 -- 172.21.15.87:0/2949146108 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65240cdbe0 con 0x7f652415c2d0 2024-09-17T03:14:16.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.816+0000 7f652891d640 1 --2- 172.21.15.87:0/2949146108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f652415c2d0 0x7f65240caa40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6514034040 tx=0x7f6514003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:16.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.817+0000 7f6519ffb640 1 -- 172.21.15.87:0/2949146108 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f651403e030 con 0x7f652415c2d0 2024-09-17T03:14:16.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.817+0000 7f6519ffb640 1 -- 172.21.15.87:0/2949146108 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65140047a0 con 0x7f652415c2d0 2024-09-17T03:14:16.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.817+0000 7f652b382640 1 -- 172.21.15.87:0/2949146108 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65240ce050 con 0x7f652415c2d0 2024-09-17T03:14:16.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.817+0000 7f6519ffb640 1 -- 172.21.15.87:0/2949146108 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6514035040 con 0x7f652415c2d0 2024-09-17T03:14:16.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.817+0000 7f652b382640 1 -- 172.21.15.87:0/2949146108 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65241f5e20 con 0x7f652415c2d0 2024-09-17T03:14:16.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.818+0000 7f652b382640 1 -- 172.21.15.87:0/2949146108 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65240b6300 con 0x7f652415c2d0 2024-09-17T03:14:16.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.818+0000 7f6519ffb640 1 -- 172.21.15.87:0/2949146108 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6514041030 con 0x7f652415c2d0 2024-09-17T03:14:16.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.819+0000 7f6519ffb640 1 --2- 172.21.15.87:0/2949146108 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6508078bc0 0x7f650807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:16.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.819+0000 7f6519ffb640 1 -- 172.21.15.87:0/2949146108 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f65140bb630 con 0x7f652415c2d0 2024-09-17T03:14:16.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.819+0000 7f652911e640 1 --2- 172.21.15.87:0/2949146108 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6508078bc0 0x7f650807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:16.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.820+0000 7f652911e640 1 --2- 172.21.15.87:0/2949146108 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6508078bc0 0x7f650807b0d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f652000ab90 tx=0x7f6520009230 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:16.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.823+0000 7f6519ffb640 1 -- 172.21.15.87:0/2949146108 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f65240b6300 con 0x7f652415c2d0 2024-09-17T03:14:16.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.997+0000 7f652b382640 1 -- 172.21.15.87:0/2949146108 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65240cddb0 con 0x7f652415c2d0 2024-09-17T03:14:16.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:16.998+0000 7f6519ffb640 1 -- 172.21.15.87:0/2949146108 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16693 (secure 0 0 0) 0x7f65240cddb0 con 0x7f652415c2d0 2024-09-17T03:14:16.997 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:16.997 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":23,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:14:15.323716+0000","last_up_change":"2024-09-17T03:14:01.833605+0000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:14:03.433404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:14:11.525547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T03:14:12.626396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6825","nonce":2430409749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6827","nonce":2430409749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6831","nonce":2430409749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6829","nonce":2430409749}]},"public_addr":"172.21.15.87:6825/2430409749","cluster_addr":"172.21.15.87:6827/2430409749","heartbeat_back_addr":"172.21.15.87:6831/2430409749","heartbeat_front_addr":"172.21.15.87:6829/2430409749","state":["exists","up"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6809","nonce":3467605435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6811","nonce":3467605435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6815","nonce":3467605435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6813","nonce":3467605435}]},"public_addr":"172.21.15.87:6809/3467605435","cluster_addr":"172.21.15.87:6811/3467605435","heartbeat_back_addr":"172.21.15.87:6815/3467605435","heartbeat_front_addr":"172.21.15.87:6813/3467605435","state":["exists","up"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6801","nonce":1450834766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6803","nonce":1450834766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6807","nonce":1450834766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6805","nonce":1450834766}]},"public_addr":"172.21.15.87:6801/1450834766","cluster_addr":"172.21.15.87:6803/1450834766","heartbeat_back_addr":"172.21.15.87:6807/1450834766","heartbeat_front_addr":"172.21.15.87:6805/1450834766","state":["exists","up"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6817","nonce":4253161216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6819","nonce":4253161216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6823","nonce":4253161216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6821","nonce":4253161216}]},"public_addr":"172.21.15.87:6817/4253161216","cluster_addr":"172.21.15.87:6819/4253161216","heartbeat_back_addr":"172.21.15.87:6823/4253161216","heartbeat_front_addr":"172.21.15.87:6821/4253161216","state":["exists","up"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6803","nonce":2414907089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6805","nonce":2414907089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6809","nonce":2414907089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6807","nonce":2414907089}]},"public_addr":"172.21.15.149:6803/2414907089","cluster_addr":"172.21.15.149:6805/2414907089","heartbeat_back_addr":"172.21.15.149:6809/2414907089","heartbeat_front_addr":"172.21.15.149:6807/2414907089","state":["exists","up"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6827","nonce":2182999530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6829","nonce":2182999530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6833","nonce":2182999530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6831","nonce":2182999530}]},"public_addr":"172.21.15.149:6827/2182999530","cluster_addr":"172.21.15.149:6829/2182999530","heartbeat_back_addr":"172.21.15.149:6833/2182999530","heartbeat_front_addr":"172.21.15.149:6831/2182999530","state":["exists","up"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6811","nonce":4289519146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6813","nonce":4289519146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6817","nonce":4289519146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6815","nonce":4289519146}]},"public_addr":"172.21.15.149:6811/4289519146","cluster_addr":"172.21.15.149:6813/4289519146","heartbeat_back_addr":"172.21.15.149:6817/4289519146","heartbeat_front_addr":"172.21.15.149:6815/4289519146","state":["exists","up"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6819","nonce":2771010477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6821","nonce":2771010477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6825","nonce":2771010477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6823","nonce":2771010477}]},"public_addr":"172.21.15.149:6819/2771010477","cluster_addr":"172.21.15.149:6821/2771010477","heartbeat_back_addr":"172.21.15.149:6825/2771010477","heartbeat_front_addr":"172.21.15.149:6823/2771010477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.313297+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.642861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.654781+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.302606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.385791+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.915266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.121365+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.394432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:14:16.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.000+0000 7f652b382640 1 -- 172.21.15.87:0/2949146108 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6508078bc0 msgr2=0x7f650807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:17.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.000+0000 7f652b382640 1 --2- 172.21.15.87:0/2949146108 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6508078bc0 0x7f650807b0d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f652000ab90 tx=0x7f6520009230 comp rx=0 tx=0).stop 2024-09-17T03:14:17.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.001+0000 7f652b382640 1 -- 172.21.15.87:0/2949146108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f652415c2d0 msgr2=0x7f65240caa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:17.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.001+0000 7f652b382640 1 --2- 172.21.15.87:0/2949146108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f652415c2d0 0x7f65240caa40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6514034040 tx=0x7f6514003970 comp rx=0 tx=0).stop 2024-09-17T03:14:17.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.001+0000 7f652b382640 1 -- 172.21.15.87:0/2949146108 shutdown_connections 2024-09-17T03:14:17.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.001+0000 7f652b382640 1 --2- 172.21.15.87:0/2949146108 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6508078bc0 0x7f650807b0d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:17.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.001+0000 7f652b382640 1 --2- 172.21.15.87:0/2949146108 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f652415ecf0 0x7f65241f6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:17.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.001+0000 7f652b382640 1 --2- 172.21.15.87:0/2949146108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f652415c2d0 0x7f65240caa40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:17.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.001+0000 7f652b382640 1 --2- 172.21.15.87:0/2949146108 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f65240b5ef0 0x7f65240cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:17.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.001+0000 7f652b382640 1 -- 172.21.15.87:0/2949146108 >> 172.21.15.87:0/2949146108 conn(0x7f65240d4470 msgr2=0x7f65240d0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:17.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.001+0000 7f652b382640 1 -- 172.21.15.87:0/2949146108 shutdown_connections 2024-09-17T03:14:17.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.001+0000 7f652b382640 1 -- 172.21.15.87:0/2949146108 wait complete. 2024-09-17T03:14:17.010 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T03:14:17.011 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:14:17.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.123+0000 7f5f7ae0e640 1 -- 172.21.15.87:0/843475605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f740b3d60 msgr2=0x7f5f740b4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:17.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.123+0000 7f5f7ae0e640 1 --2- 172.21.15.87:0/843475605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f740b3d60 0x7f5f740b4170 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5f740cdfa0 tx=0x7f5f6802a4d0 comp rx=0 tx=0).stop 2024-09-17T03:14:17.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.123+0000 7f5f7ae0e640 1 -- 172.21.15.87:0/843475605 shutdown_connections 2024-09-17T03:14:17.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.123+0000 7f5f7ae0e640 1 --2- 172.21.15.87:0/843475605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f740b3d60 0x7f5f740b4170 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:17.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.123+0000 7f5f7ae0e640 1 --2- 172.21.15.87:0/843475605 >> v2:172.21.15.149:3300/0 conn(0x7f5f740710e0 0x7f5f740714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:17.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.123+0000 7f5f7ae0e640 1 --2- 172.21.15.87:0/843475605 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5f740b7f10 0x7f5f7415c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:17.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.123+0000 7f5f7ae0e640 1 -- 172.21.15.87:0/843475605 >> 172.21.15.87:0/843475605 conn(0x7f5f74069330 msgr2=0x7f5f740ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:17.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.123+0000 7f5f7ae0e640 1 -- 172.21.15.87:0/843475605 shutdown_connections 2024-09-17T03:14:17.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.123+0000 7f5f7ae0e640 1 -- 172.21.15.87:0/843475605 wait complete. 2024-09-17T03:14:17.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.125+0000 7f5f7ae0e640 1 Processor -- start 2024-09-17T03:14:17.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.125+0000 7f5f7ae0e640 1 -- start start 2024-09-17T03:14:17.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.126+0000 7f5f7ae0e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f740710e0 0x7f5f741f3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:17.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.126+0000 7f5f7ae0e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f740b3d60 0x7f5f740ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:17.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.126+0000 7f5f7ae0e640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5f740b7f10 0x7f5f740ab3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:17.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.126+0000 7f5f7ae0e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5f740b7c90 con 0x7f5f740710e0 2024-09-17T03:14:17.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.126+0000 7f5f7ae0e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5f740b7b10 con 0x7f5f740b3d60 2024-09-17T03:14:17.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.126+0000 7f5f7ae0e640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5f74071a30 con 0x7f5f740b7f10 2024-09-17T03:14:17.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.126+0000 7f5f78baa640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f740710e0 0x7f5f741f3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:17.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.126+0000 7f5f78baa640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f740710e0 0x7f5f741f3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39928/0 (socket says 172.21.15.87:39928) 2024-09-17T03:14:17.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.126+0000 7f5f78baa640 1 -- 172.21.15.87:0/3095955764 learned_addr learned my addr 172.21.15.87:0/3095955764 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:17.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.126+0000 7f5f73fff640 1 --2- 172.21.15.87:0/3095955764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f740b3d60 0x7f5f740ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:17.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.126+0000 7f5f793ab640 1 --2- 172.21.15.87:0/3095955764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5f740b7f10 0x7f5f740ab3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:17.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.126+0000 7f5f78baa640 1 -- 172.21.15.87:0/3095955764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5f740b7f10 msgr2=0x7f5f740ab3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:17.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.126+0000 7f5f78baa640 1 --2- 172.21.15.87:0/3095955764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5f740b7f10 0x7f5f740ab3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:17.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.126+0000 7f5f78baa640 1 -- 172.21.15.87:0/3095955764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f740b3d60 msgr2=0x7f5f740ca9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:17.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.126+0000 7f5f78baa640 1 --2- 172.21.15.87:0/3095955764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f740b3d60 0x7f5f740ca9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:17.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.126+0000 7f5f78baa640 1 -- 172.21.15.87:0/3095955764 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f740c8e10 con 0x7f5f740710e0 2024-09-17T03:14:17.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.127+0000 7f5f78baa640 1 --2- 172.21.15.87:0/3095955764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f740710e0 0x7f5f741f3140 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400d740 tx=0x7f5f6400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:17.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.127+0000 7f5f717fa640 1 -- 172.21.15.87:0/3095955764 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f64017070 con 0x7f5f740710e0 2024-09-17T03:14:17.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.127+0000 7f5f717fa640 1 -- 172.21.15.87:0/3095955764 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f64009d20 con 0x7f5f740710e0 2024-09-17T03:14:17.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.127+0000 7f5f717fa640 1 -- 172.21.15.87:0/3095955764 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f640106c0 con 0x7f5f740710e0 2024-09-17T03:14:17.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.127+0000 7f5f7ae0e640 1 -- 172.21.15.87:0/3095955764 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f740a94a0 con 0x7f5f740710e0 2024-09-17T03:14:17.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.127+0000 7f5f7ae0e640 1 -- 172.21.15.87:0/3095955764 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f741f3f50 con 0x7f5f740710e0 2024-09-17T03:14:17.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.128+0000 7f5f7ae0e640 1 -- 172.21.15.87:0/3095955764 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f740cda80 con 0x7f5f740710e0 2024-09-17T03:14:17.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.128+0000 7f5f717fa640 1 -- 172.21.15.87:0/3095955764 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5f6401a070 con 0x7f5f740710e0 2024-09-17T03:14:17.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.129+0000 7f5f717fa640 1 --2- 172.21.15.87:0/3095955764 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5f60078b90 0x7f5f6007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:17.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.129+0000 7f5f717fa640 1 -- 172.21.15.87:0/3095955764 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f5f6409af10 con 0x7f5f740710e0 2024-09-17T03:14:17.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.129+0000 7f5f73fff640 1 --2- 172.21.15.87:0/3095955764 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5f60078b90 0x7f5f6007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:17.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.129+0000 7f5f73fff640 1 --2- 172.21.15.87:0/3095955764 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5f60078b90 0x7f5f6007b0a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5f68033000 tx=0x7f5f68004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:17.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.133+0000 7f5f717fa640 1 -- 172.21.15.87:0/3095955764 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f5f740cda80 con 0x7f5f740710e0 2024-09-17T03:14:17.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.305+0000 7f5f7ae0e640 1 -- 172.21.15.87:0/3095955764 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5f7415b4c0 con 0x7f5f60078b90 2024-09-17T03:14:17.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.310+0000 7f5f717fa640 1 -- 172.21.15.87:0/3095955764 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241501 (secure 0 0 0) 0x7f5f68005600 con 0x7f5f60078b90 2024-09-17T03:14:17.311 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:17.312 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-09-17T03:14:17.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.313+0000 7f5f7ae0e640 1 -- 172.21.15.87:0/3095955764 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5f60078b90 msgr2=0x7f5f6007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:17.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.313+0000 7f5f7ae0e640 1 --2- 172.21.15.87:0/3095955764 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5f60078b90 0x7f5f6007b0a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5f68033000 tx=0x7f5f68004730 comp rx=0 tx=0).stop 2024-09-17T03:14:17.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.313+0000 7f5f7ae0e640 1 -- 172.21.15.87:0/3095955764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f740710e0 msgr2=0x7f5f741f3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:17.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.313+0000 7f5f7ae0e640 1 --2- 172.21.15.87:0/3095955764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f740710e0 0x7f5f741f3140 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400d740 tx=0x7f5f6400dc00 comp rx=0 tx=0).stop 2024-09-17T03:14:17.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.313+0000 7f5f7ae0e640 1 -- 172.21.15.87:0/3095955764 shutdown_connections 2024-09-17T03:14:17.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.313+0000 7f5f7ae0e640 1 --2- 172.21.15.87:0/3095955764 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5f60078b90 0x7f5f6007b0a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:17.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.313+0000 7f5f7ae0e640 1 --2- 172.21.15.87:0/3095955764 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5f740b7f10 0x7f5f740ab3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:17.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.313+0000 7f5f7ae0e640 1 --2- 172.21.15.87:0/3095955764 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f740b3d60 0x7f5f740ca9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:17.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.313+0000 7f5f7ae0e640 1 --2- 172.21.15.87:0/3095955764 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f740710e0 0x7f5f741f3140 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:17.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.313+0000 7f5f7ae0e640 1 -- 172.21.15.87:0/3095955764 >> 172.21.15.87:0/3095955764 conn(0x7f5f74069330 msgr2=0x7f5f740b5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:17.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.313+0000 7f5f7ae0e640 1 -- 172.21.15.87:0/3095955764 shutdown_connections 2024-09-17T03:14:17.315 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.313+0000 7f5f7ae0e640 1 -- 172.21.15.87:0/3095955764 wait complete. 2024-09-17T03:14:17.322 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-17T03:14:15.416880+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":120,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":80,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219424,"kb_used_data":2668,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755296,"statfs":{"total":773094113280,"available":772869423104,"internally_reserved":0,"allocated":2732032,"data_stored":1663224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001001"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.344245+0000","last_change":"2024-09-17T03:14:14.338758+0000","last_active":"2024-09-17T03:14:14.344245+0000","last_peered":"2024-09-17T03:14:14.344245+0000","last_clean":"2024-09-17T03:14:14.344245+0000","last_became_active":"2024-09-17T03:14:14.337821+0000","last_became_peered":"2024-09-17T03:14:14.337821+0000","last_unstale":"2024-09-17T03:14:14.344245+0000","last_undegraded":"2024-09-17T03:14:14.344245+0000","last_fullsized":"2024-09-17T03:14:14.344245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:47:29.951060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.307401+0000","last_change":"2024-09-17T03:14:12.307401+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.307401+0000","last_undegraded":"2024-09-17T03:14:12.307401+0000","last_fullsized":"2024-09-17T03:14:12.307401+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.338533+0000","last_change":"2024-09-17T03:14:14.330562+0000","last_active":"2024-09-17T03:14:14.338533+0000","last_peered":"2024-09-17T03:14:14.338533+0000","last_clean":"2024-09-17T03:14:14.338533+0000","last_became_active":"2024-09-17T03:14:14.330255+0000","last_became_peered":"2024-09-17T03:14:14.330255+0000","last_unstale":"2024-09-17T03:14:14.338533+0000","last_undegraded":"2024-09-17T03:14:14.338533+0000","last_fullsized":"2024-09-17T03:14:14.338533+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:17.454865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.311093+0000","last_change":"2024-09-17T03:14:12.311093+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.311093+0000","last_undegraded":"2024-09-17T03:14:12.311093+0000","last_fullsized":"2024-09-17T03:14:12.311093+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.309399+0000","last_change":"2024-09-17T03:14:12.309399+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.309399+0000","last_undegraded":"2024-09-17T03:14:12.309399+0000","last_fullsized":"2024-09-17T03:14:12.309399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.311069+0000","last_change":"2024-09-17T03:14:12.311069+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.311069+0000","last_undegraded":"2024-09-17T03:14:12.311069+0000","last_fullsized":"2024-09-17T03:14:12.311069+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.344304+0000","last_change":"2024-09-17T03:14:14.339215+0000","last_active":"2024-09-17T03:14:14.344304+0000","last_peered":"2024-09-17T03:14:14.344304+0000","last_clean":"2024-09-17T03:14:14.344304+0000","last_became_active":"2024-09-17T03:14:14.337738+0000","last_became_peered":"2024-09-17T03:14:14.337738+0000","last_unstale":"2024-09-17T03:14:14.344304+0000","last_undegraded":"2024-09-17T03:14:14.344304+0000","last_fullsized":"2024-09-17T03:14:14.344304+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:58.797420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.302832+0000","last_change":"2024-09-17T03:14:12.302832+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.302832+0000","last_undegraded":"2024-09-17T03:14:12.302832+0000","last_fullsized":"2024-09-17T03:14:12.302832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.308447+0000","last_change":"2024-09-17T03:14:12.308447+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.308447+0000","last_undegraded":"2024-09-17T03:14:12.308447+0000","last_fullsized":"2024-09-17T03:14:12.308447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.313764+0000","last_change":"2024-09-17T03:14:12.313764+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.313764+0000","last_undegraded":"2024-09-17T03:14:12.313764+0000","last_fullsized":"2024-09-17T03:14:12.313764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.311133+0000","last_change":"2024-09-17T03:14:12.311133+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.311133+0000","last_undegraded":"2024-09-17T03:14:12.311133+0000","last_fullsized":"2024-09-17T03:14:12.311133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.313748+0000","last_change":"2024-09-17T03:14:12.313748+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.313748+0000","last_undegraded":"2024-09-17T03:14:12.313748+0000","last_fullsized":"2024-09-17T03:14:12.313748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.310657+0000","last_change":"2024-09-17T03:14:12.310657+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.310657+0000","last_undegraded":"2024-09-17T03:14:12.310657+0000","last_fullsized":"2024-09-17T03:14:12.310657+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.308595+0000","last_change":"2024-09-17T03:14:12.308595+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.308595+0000","last_undegraded":"2024-09-17T03:14:12.308595+0000","last_fullsized":"2024-09-17T03:14:12.308595+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.345087+0000","last_change":"2024-09-17T03:14:13.321317+0000","last_active":"2024-09-17T03:14:14.345087+0000","last_peered":"2024-09-17T03:14:14.345087+0000","last_clean":"2024-09-17T03:14:14.345087+0000","last_became_active":"2024-09-17T03:14:13.320995+0000","last_became_peered":"2024-09-17T03:14:13.320995+0000","last_unstale":"2024-09-17T03:14:14.345087+0000","last_undegraded":"2024-09-17T03:14:14.345087+0000","last_fullsized":"2024-09-17T03:14:14.345087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:56.906875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.308531+0000","last_change":"2024-09-17T03:14:12.308531+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.308531+0000","last_undegraded":"2024-09-17T03:14:12.308531+0000","last_fullsized":"2024-09-17T03:14:12.308531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.345671+0000","last_change":"2024-09-17T03:14:13.322488+0000","last_active":"2024-09-17T03:14:14.345671+0000","last_peered":"2024-09-17T03:14:14.345671+0000","last_clean":"2024-09-17T03:14:14.345671+0000","last_became_active":"2024-09-17T03:14:13.322170+0000","last_became_peered":"2024-09-17T03:14:13.322170+0000","last_unstale":"2024-09-17T03:14:14.345671+0000","last_undegraded":"2024-09-17T03:14:14.345671+0000","last_fullsized":"2024-09-17T03:14:14.345671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:13.679632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.306379+0000","last_change":"2024-09-17T03:14:12.306379+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.306379+0000","last_undegraded":"2024-09-17T03:14:12.306379+0000","last_fullsized":"2024-09-17T03:14:12.306379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.309393+0000","last_change":"2024-09-17T03:14:12.309393+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.309393+0000","last_undegraded":"2024-09-17T03:14:12.309393+0000","last_fullsized":"2024-09-17T03:14:12.309393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":105,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T03:14:12.301021+0000","last_change":"2024-09-17T03:14:04.862167+0000","last_active":"2024-09-17T03:14:12.301021+0000","last_peered":"2024-09-17T03:14:12.301021+0000","last_clean":"2024-09-17T03:14:12.301021+0000","last_became_active":"2024-09-17T03:14:04.861731+0000","last_became_peered":"2024-09-17T03:14:04.861731+0000","last_unstale":"2024-09-17T03:14:12.301021+0000","last_undegraded":"2024-09-17T03:14:12.301021+0000","last_fullsized":"2024-09-17T03:14:12.301021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_clean_scrub_stamp":"2024-09-17T03:14:03.847721+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:32.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.311265+0000","last_change":"2024-09-17T03:14:12.311265+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.311265+0000","last_undegraded":"2024-09-17T03:14:12.311265+0000","last_fullsized":"2024-09-17T03:14:12.311265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.338243+0000","last_change":"2024-09-17T03:14:14.331745+0000","last_active":"2024-09-17T03:14:14.338243+0000","last_peered":"2024-09-17T03:14:14.338243+0000","last_clean":"2024-09-17T03:14:14.338243+0000","last_became_active":"2024-09-17T03:14:14.331207+0000","last_became_peered":"2024-09-17T03:14:14.331207+0000","last_unstale":"2024-09-17T03:14:14.338243+0000","last_undegraded":"2024-09-17T03:14:14.338243+0000","last_fullsized":"2024-09-17T03:14:14.338243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:32.588857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.309532+0000","last_change":"2024-09-17T03:14:12.309532+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.309532+0000","last_undegraded":"2024-09-17T03:14:12.309532+0000","last_fullsized":"2024-09-17T03:14:12.309532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.313614+0000","last_change":"2024-09-17T03:14:12.313614+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.313614+0000","last_undegraded":"2024-09-17T03:14:12.313614+0000","last_fullsized":"2024-09-17T03:14:12.313614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.310047+0000","last_change":"2024-09-17T03:14:12.310047+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.310047+0000","last_undegraded":"2024-09-17T03:14:12.310047+0000","last_fullsized":"2024-09-17T03:14:12.310047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.306474+0000","last_change":"2024-09-17T03:14:12.306474+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.306474+0000","last_undegraded":"2024-09-17T03:14:12.306474+0000","last_fullsized":"2024-09-17T03:14:12.306474+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.309954+0000","last_change":"2024-09-17T03:14:12.309954+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.309954+0000","last_undegraded":"2024-09-17T03:14:12.309954+0000","last_fullsized":"2024-09-17T03:14:12.309954+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.321286+0000","last_change":"2024-09-17T03:14:13.321756+0000","last_active":"2024-09-17T03:14:14.321286+0000","last_peered":"2024-09-17T03:14:14.321286+0000","last_clean":"2024-09-17T03:14:14.321286+0000","last_became_active":"2024-09-17T03:14:13.321392+0000","last_became_peered":"2024-09-17T03:14:13.321392+0000","last_unstale":"2024-09-17T03:14:14.321286+0000","last_undegraded":"2024-09-17T03:14:14.321286+0000","last_fullsized":"2024-09-17T03:14:14.321286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:33.594511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.346129+0000","last_change":"2024-09-17T03:14:14.341909+0000","last_active":"2024-09-17T03:14:14.346129+0000","last_peered":"2024-09-17T03:14:14.346129+0000","last_clean":"2024-09-17T03:14:14.346129+0000","last_became_active":"2024-09-17T03:14:14.341571+0000","last_became_peered":"2024-09-17T03:14:14.341571+0000","last_unstale":"2024-09-17T03:14:14.346129+0000","last_undegraded":"2024-09-17T03:14:14.346129+0000","last_fullsized":"2024-09-17T03:14:14.346129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:28.494598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.310378+0000","last_change":"2024-09-17T03:14:12.310378+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.310378+0000","last_undegraded":"2024-09-17T03:14:12.310378+0000","last_fullsized":"2024-09-17T03:14:12.310378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.344893+0000","last_change":"2024-09-17T03:14:14.340502+0000","last_active":"2024-09-17T03:14:14.344893+0000","last_peered":"2024-09-17T03:14:14.344893+0000","last_clean":"2024-09-17T03:14:14.344893+0000","last_became_active":"2024-09-17T03:14:14.340215+0000","last_became_peered":"2024-09-17T03:14:14.340215+0000","last_unstale":"2024-09-17T03:14:14.344893+0000","last_undegraded":"2024-09-17T03:14:14.344893+0000","last_fullsized":"2024-09-17T03:14:14.344893+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:46:40.874255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.308458+0000","last_change":"2024-09-17T03:14:12.308458+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.308458+0000","last_undegraded":"2024-09-17T03:14:12.308458+0000","last_fullsized":"2024-09-17T03:14:12.308458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.320414+0000","last_change":"2024-09-17T03:14:13.319558+0000","last_active":"2024-09-17T03:14:14.320414+0000","last_peered":"2024-09-17T03:14:14.320414+0000","last_clean":"2024-09-17T03:14:14.320414+0000","last_became_active":"2024-09-17T03:14:13.319236+0000","last_became_peered":"2024-09-17T03:14:13.319236+0000","last_unstale":"2024-09-17T03:14:14.320414+0000","last_undegraded":"2024-09-17T03:14:14.320414+0000","last_fullsized":"2024-09-17T03:14:14.320414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:25:39.260408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.337352+0000","last_change":"2024-09-17T03:14:14.329694+0000","last_active":"2024-09-17T03:14:14.337352+0000","last_peered":"2024-09-17T03:14:14.337352+0000","last_clean":"2024-09-17T03:14:14.337352+0000","last_became_active":"2024-09-17T03:14:14.329289+0000","last_became_peered":"2024-09-17T03:14:14.329289+0000","last_unstale":"2024-09-17T03:14:14.337352+0000","last_undegraded":"2024-09-17T03:14:14.337352+0000","last_fullsized":"2024-09-17T03:14:14.337352+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:55.345670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.309458+0000","last_change":"2024-09-17T03:14:12.309458+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.309458+0000","last_undegraded":"2024-09-17T03:14:12.309458+0000","last_fullsized":"2024-09-17T03:14:12.309458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.311838+0000","last_change":"2024-09-17T03:14:12.311838+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.311838+0000","last_undegraded":"2024-09-17T03:14:12.311838+0000","last_fullsized":"2024-09-17T03:14:12.311838+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.308581+0000","last_change":"2024-09-17T03:14:12.308581+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.308581+0000","last_undegraded":"2024-09-17T03:14:12.308581+0000","last_fullsized":"2024-09-17T03:14:12.308581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.308533+0000","last_change":"2024-09-17T03:14:12.308533+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.308533+0000","last_undegraded":"2024-09-17T03:14:12.308533+0000","last_fullsized":"2024-09-17T03:14:12.308533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.307080+0000","last_change":"2024-09-17T03:14:12.307080+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.307080+0000","last_undegraded":"2024-09-17T03:14:12.307080+0000","last_fullsized":"2024-09-17T03:14:12.307080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.351905+0000","last_change":"2024-09-17T03:14:13.324341+0000","last_active":"2024-09-17T03:14:14.351905+0000","last_peered":"2024-09-17T03:14:14.351905+0000","last_clean":"2024-09-17T03:14:14.351905+0000","last_became_active":"2024-09-17T03:14:13.323959+0000","last_became_peered":"2024-09-17T03:14:13.323959+0000","last_unstale":"2024-09-17T03:14:14.351905+0000","last_undegraded":"2024-09-17T03:14:14.351905+0000","last_fullsized":"2024-09-17T03:14:14.351905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:18:06.456762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14,"ondisk_log_size":14,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":56940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":184320,"data_stored":56940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":778240,"data_stored":647308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":196608,"data_stored":62757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":184320,"data_stored":56940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":778240,"data_stored":647308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":56940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":78091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:14:17.327 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:14:17.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.476+0000 7fe455039640 1 -- 172.21.15.87:0/1725418733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe43c003a30 msgr2=0x7fe43c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:17.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.476+0000 7fe455039640 1 --2- 172.21.15.87:0/1725418733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe43c003a30 0x7fe43c003e60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe440001730 tx=0x7fe43c005830 comp rx=0 tx=0).stop 2024-09-17T03:14:17.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.476+0000 7fe44f59e640 1 -- 172.21.15.87:0/1725418733 reap_dead start 2024-09-17T03:14:17.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.476+0000 7fe455039640 1 -- 172.21.15.87:0/1725418733 shutdown_connections 2024-09-17T03:14:17.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.476+0000 7fe455039640 1 -- 172.21.15.87:0/1725418733 >> 172.21.15.87:0/1725418733 conn(0x7fe4500d4470 msgr2=0x7fe4500d4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:17.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.476+0000 7fe455039640 1 -- 172.21.15.87:0/1725418733 shutdown_connections 2024-09-17T03:14:17.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.476+0000 7fe455039640 1 -- 172.21.15.87:0/1725418733 wait complete. 2024-09-17T03:14:17.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.478+0000 7fe455039640 1 Processor -- start 2024-09-17T03:14:17.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.478+0000 7fe455039640 1 -- start start 2024-09-17T03:14:17.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.480+0000 7fe455039640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe43c003a30 0x7fe450151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:17.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.480+0000 7fe455039640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4501f53d0 0x7fe450156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:17.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.480+0000 7fe455039640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe450156ab0 0x7fe4500cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:17.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.480+0000 7fe455039640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe45015f510 con 0x7fe450156ab0 2024-09-17T03:14:17.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.480+0000 7fe455039640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe45015f390 con 0x7fe4501f53d0 2024-09-17T03:14:17.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.480+0000 7fe455039640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe4500d2c30 con 0x7fe43c003a30 2024-09-17T03:14:17.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.480+0000 7fe44f59e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe450156ab0 0x7fe4500cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:17.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.480+0000 7fe44f59e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe450156ab0 0x7fe4500cb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39948/0 (socket says 172.21.15.87:39948) 2024-09-17T03:14:17.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.480+0000 7fe44f59e640 1 -- 172.21.15.87:0/769126565 learned_addr learned my addr 172.21.15.87:0/769126565 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:17.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.480+0000 7fe44ed9d640 1 --2- 172.21.15.87:0/769126565 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe43c003a30 0x7fe450151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:17.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.480+0000 7fe44659c640 1 --2- 172.21.15.87:0/769126565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4501f53d0 0x7fe450156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:17.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.480+0000 7fe44f59e640 1 -- 172.21.15.87:0/769126565 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe43c003a30 msgr2=0x7fe450151b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:17.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.480+0000 7fe44f59e640 1 --2- 172.21.15.87:0/769126565 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe43c003a30 0x7fe450151b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:17.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.480+0000 7fe44f59e640 1 -- 172.21.15.87:0/769126565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4501f53d0 msgr2=0x7fe450156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:17.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.480+0000 7fe44f59e640 1 --2- 172.21.15.87:0/769126565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4501f53d0 0x7fe450156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:17.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.480+0000 7fe44f59e640 1 -- 172.21.15.87:0/769126565 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe450156ec0 con 0x7fe450156ab0 2024-09-17T03:14:17.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.480+0000 7fe44f59e640 1 --2- 172.21.15.87:0/769126565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe450156ab0 0x7fe4500cb4b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe448001330 tx=0x7fe44800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:17.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.480+0000 7fe447fff640 1 -- 172.21.15.87:0/769126565 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44800cbc0 con 0x7fe450156ab0 2024-09-17T03:14:17.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.480+0000 7fe447fff640 1 -- 172.21.15.87:0/769126565 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe448018070 con 0x7fe450156ab0 2024-09-17T03:14:17.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.480+0000 7fe455039640 1 -- 172.21.15.87:0/769126565 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4501f5db0 con 0x7fe450156ab0 2024-09-17T03:14:17.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.480+0000 7fe447fff640 1 -- 172.21.15.87:0/769126565 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4480116c0 con 0x7fe450156ab0 2024-09-17T03:14:17.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.481+0000 7fe455039640 1 -- 172.21.15.87:0/769126565 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4500ccac0 con 0x7fe450156ab0 2024-09-17T03:14:17.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.482+0000 7fe455039640 1 -- 172.21.15.87:0/769126565 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4500cf840 con 0x7fe450156ab0 2024-09-17T03:14:17.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.482+0000 7fe447fff640 1 -- 172.21.15.87:0/769126565 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe448011860 con 0x7fe450156ab0 2024-09-17T03:14:17.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.483+0000 7fe447fff640 1 --2- 172.21.15.87:0/769126565 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe430078bc0 0x7fe43007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:17.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.483+0000 7fe447fff640 1 -- 172.21.15.87:0/769126565 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe44809b090 con 0x7fe450156ab0 2024-09-17T03:14:17.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.483+0000 7fe44ed9d640 1 --2- 172.21.15.87:0/769126565 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe430078bc0 0x7fe43007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:17.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.483+0000 7fe44ed9d640 1 --2- 172.21.15.87:0/769126565 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe430078bc0 0x7fe43007b0d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe440032000 tx=0x7fe440001270 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:17.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.487+0000 7fe447fff640 1 -- 172.21.15.87:0/769126565 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe450156ec0 con 0x7fe450156ab0 2024-09-17T03:14:17.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.658+0000 7fe455039640 1 -- 172.21.15.87:0/769126565 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe4500ce6f0 con 0x7fe430078bc0 2024-09-17T03:14:17.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.662+0000 7fe447fff640 1 -- 172.21.15.87:0/769126565 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241501 (secure 0 0 0) 0x7fe4500ce6f0 con 0x7fe430078bc0 2024-09-17T03:14:17.663 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:17.664 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-09-17T03:14:17.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.665+0000 7fe455039640 1 -- 172.21.15.87:0/769126565 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe430078bc0 msgr2=0x7fe43007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:17.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.665+0000 7fe455039640 1 --2- 172.21.15.87:0/769126565 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe430078bc0 0x7fe43007b0d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe440032000 tx=0x7fe440001270 comp rx=0 tx=0).stop 2024-09-17T03:14:17.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.665+0000 7fe455039640 1 -- 172.21.15.87:0/769126565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe450156ab0 msgr2=0x7fe4500cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:17.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.665+0000 7fe455039640 1 --2- 172.21.15.87:0/769126565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe450156ab0 0x7fe4500cb4b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe448001330 tx=0x7fe44800ed90 comp rx=0 tx=0).stop 2024-09-17T03:14:17.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.665+0000 7fe455039640 1 -- 172.21.15.87:0/769126565 shutdown_connections 2024-09-17T03:14:17.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.665+0000 7fe455039640 1 --2- 172.21.15.87:0/769126565 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe430078bc0 0x7fe43007b0d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:17.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.665+0000 7fe455039640 1 --2- 172.21.15.87:0/769126565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe450156ab0 0x7fe4500cb4b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:17.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.665+0000 7fe455039640 1 --2- 172.21.15.87:0/769126565 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4501f53d0 0x7fe450156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:17.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.665+0000 7fe455039640 1 --2- 172.21.15.87:0/769126565 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe43c003a30 0x7fe450151b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:17.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.666+0000 7fe455039640 1 -- 172.21.15.87:0/769126565 >> 172.21.15.87:0/769126565 conn(0x7fe4500d4470 msgr2=0x7fe4500d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:17.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.666+0000 7fe455039640 1 -- 172.21.15.87:0/769126565 shutdown_connections 2024-09-17T03:14:17.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.666+0000 7fe455039640 1 -- 172.21.15.87:0/769126565 wait complete. 2024-09-17T03:14:17.675 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-17T03:14:17.417421+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":120,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":80,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219424,"kb_used_data":2668,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755296,"statfs":{"total":773094113280,"available":772869423104,"internally_reserved":0,"allocated":2732032,"data_stored":1663224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.114782"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.344245+0000","last_change":"2024-09-17T03:14:14.338758+0000","last_active":"2024-09-17T03:14:14.344245+0000","last_peered":"2024-09-17T03:14:14.344245+0000","last_clean":"2024-09-17T03:14:14.344245+0000","last_became_active":"2024-09-17T03:14:14.337821+0000","last_became_peered":"2024-09-17T03:14:14.337821+0000","last_unstale":"2024-09-17T03:14:14.344245+0000","last_undegraded":"2024-09-17T03:14:14.344245+0000","last_fullsized":"2024-09-17T03:14:14.344245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:47:29.951060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.307401+0000","last_change":"2024-09-17T03:14:12.307401+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.307401+0000","last_undegraded":"2024-09-17T03:14:12.307401+0000","last_fullsized":"2024-09-17T03:14:12.307401+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.338533+0000","last_change":"2024-09-17T03:14:14.330562+0000","last_active":"2024-09-17T03:14:14.338533+0000","last_peered":"2024-09-17T03:14:14.338533+0000","last_clean":"2024-09-17T03:14:14.338533+0000","last_became_active":"2024-09-17T03:14:14.330255+0000","last_became_peered":"2024-09-17T03:14:14.330255+0000","last_unstale":"2024-09-17T03:14:14.338533+0000","last_undegraded":"2024-09-17T03:14:14.338533+0000","last_fullsized":"2024-09-17T03:14:14.338533+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:17.454865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.311093+0000","last_change":"2024-09-17T03:14:12.311093+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.311093+0000","last_undegraded":"2024-09-17T03:14:12.311093+0000","last_fullsized":"2024-09-17T03:14:12.311093+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.309399+0000","last_change":"2024-09-17T03:14:12.309399+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.309399+0000","last_undegraded":"2024-09-17T03:14:12.309399+0000","last_fullsized":"2024-09-17T03:14:12.309399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.311069+0000","last_change":"2024-09-17T03:14:12.311069+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.311069+0000","last_undegraded":"2024-09-17T03:14:12.311069+0000","last_fullsized":"2024-09-17T03:14:12.311069+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.344304+0000","last_change":"2024-09-17T03:14:14.339215+0000","last_active":"2024-09-17T03:14:14.344304+0000","last_peered":"2024-09-17T03:14:14.344304+0000","last_clean":"2024-09-17T03:14:14.344304+0000","last_became_active":"2024-09-17T03:14:14.337738+0000","last_became_peered":"2024-09-17T03:14:14.337738+0000","last_unstale":"2024-09-17T03:14:14.344304+0000","last_undegraded":"2024-09-17T03:14:14.344304+0000","last_fullsized":"2024-09-17T03:14:14.344304+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:58.797420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.302832+0000","last_change":"2024-09-17T03:14:12.302832+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.302832+0000","last_undegraded":"2024-09-17T03:14:12.302832+0000","last_fullsized":"2024-09-17T03:14:12.302832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.308447+0000","last_change":"2024-09-17T03:14:12.308447+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.308447+0000","last_undegraded":"2024-09-17T03:14:12.308447+0000","last_fullsized":"2024-09-17T03:14:12.308447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.313764+0000","last_change":"2024-09-17T03:14:12.313764+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.313764+0000","last_undegraded":"2024-09-17T03:14:12.313764+0000","last_fullsized":"2024-09-17T03:14:12.313764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.311133+0000","last_change":"2024-09-17T03:14:12.311133+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.311133+0000","last_undegraded":"2024-09-17T03:14:12.311133+0000","last_fullsized":"2024-09-17T03:14:12.311133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.313748+0000","last_change":"2024-09-17T03:14:12.313748+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.313748+0000","last_undegraded":"2024-09-17T03:14:12.313748+0000","last_fullsized":"2024-09-17T03:14:12.313748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.310657+0000","last_change":"2024-09-17T03:14:12.310657+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.310657+0000","last_undegraded":"2024-09-17T03:14:12.310657+0000","last_fullsized":"2024-09-17T03:14:12.310657+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.308595+0000","last_change":"2024-09-17T03:14:12.308595+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.308595+0000","last_undegraded":"2024-09-17T03:14:12.308595+0000","last_fullsized":"2024-09-17T03:14:12.308595+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.345087+0000","last_change":"2024-09-17T03:14:13.321317+0000","last_active":"2024-09-17T03:14:14.345087+0000","last_peered":"2024-09-17T03:14:14.345087+0000","last_clean":"2024-09-17T03:14:14.345087+0000","last_became_active":"2024-09-17T03:14:13.320995+0000","last_became_peered":"2024-09-17T03:14:13.320995+0000","last_unstale":"2024-09-17T03:14:14.345087+0000","last_undegraded":"2024-09-17T03:14:14.345087+0000","last_fullsized":"2024-09-17T03:14:14.345087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:56.906875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.308531+0000","last_change":"2024-09-17T03:14:12.308531+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.308531+0000","last_undegraded":"2024-09-17T03:14:12.308531+0000","last_fullsized":"2024-09-17T03:14:12.308531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.345671+0000","last_change":"2024-09-17T03:14:13.322488+0000","last_active":"2024-09-17T03:14:14.345671+0000","last_peered":"2024-09-17T03:14:14.345671+0000","last_clean":"2024-09-17T03:14:14.345671+0000","last_became_active":"2024-09-17T03:14:13.322170+0000","last_became_peered":"2024-09-17T03:14:13.322170+0000","last_unstale":"2024-09-17T03:14:14.345671+0000","last_undegraded":"2024-09-17T03:14:14.345671+0000","last_fullsized":"2024-09-17T03:14:14.345671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:13.679632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.306379+0000","last_change":"2024-09-17T03:14:12.306379+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.306379+0000","last_undegraded":"2024-09-17T03:14:12.306379+0000","last_fullsized":"2024-09-17T03:14:12.306379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.309393+0000","last_change":"2024-09-17T03:14:12.309393+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.309393+0000","last_undegraded":"2024-09-17T03:14:12.309393+0000","last_fullsized":"2024-09-17T03:14:12.309393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":105,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T03:14:12.301021+0000","last_change":"2024-09-17T03:14:04.862167+0000","last_active":"2024-09-17T03:14:12.301021+0000","last_peered":"2024-09-17T03:14:12.301021+0000","last_clean":"2024-09-17T03:14:12.301021+0000","last_became_active":"2024-09-17T03:14:04.861731+0000","last_became_peered":"2024-09-17T03:14:04.861731+0000","last_unstale":"2024-09-17T03:14:12.301021+0000","last_undegraded":"2024-09-17T03:14:12.301021+0000","last_fullsized":"2024-09-17T03:14:12.301021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_clean_scrub_stamp":"2024-09-17T03:14:03.847721+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:32.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.311265+0000","last_change":"2024-09-17T03:14:12.311265+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.311265+0000","last_undegraded":"2024-09-17T03:14:12.311265+0000","last_fullsized":"2024-09-17T03:14:12.311265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.338243+0000","last_change":"2024-09-17T03:14:14.331745+0000","last_active":"2024-09-17T03:14:14.338243+0000","last_peered":"2024-09-17T03:14:14.338243+0000","last_clean":"2024-09-17T03:14:14.338243+0000","last_became_active":"2024-09-17T03:14:14.331207+0000","last_became_peered":"2024-09-17T03:14:14.331207+0000","last_unstale":"2024-09-17T03:14:14.338243+0000","last_undegraded":"2024-09-17T03:14:14.338243+0000","last_fullsized":"2024-09-17T03:14:14.338243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:32.588857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.309532+0000","last_change":"2024-09-17T03:14:12.309532+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.309532+0000","last_undegraded":"2024-09-17T03:14:12.309532+0000","last_fullsized":"2024-09-17T03:14:12.309532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.313614+0000","last_change":"2024-09-17T03:14:12.313614+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.313614+0000","last_undegraded":"2024-09-17T03:14:12.313614+0000","last_fullsized":"2024-09-17T03:14:12.313614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.310047+0000","last_change":"2024-09-17T03:14:12.310047+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.310047+0000","last_undegraded":"2024-09-17T03:14:12.310047+0000","last_fullsized":"2024-09-17T03:14:12.310047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.306474+0000","last_change":"2024-09-17T03:14:12.306474+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.306474+0000","last_undegraded":"2024-09-17T03:14:12.306474+0000","last_fullsized":"2024-09-17T03:14:12.306474+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.309954+0000","last_change":"2024-09-17T03:14:12.309954+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.309954+0000","last_undegraded":"2024-09-17T03:14:12.309954+0000","last_fullsized":"2024-09-17T03:14:12.309954+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.321286+0000","last_change":"2024-09-17T03:14:13.321756+0000","last_active":"2024-09-17T03:14:14.321286+0000","last_peered":"2024-09-17T03:14:14.321286+0000","last_clean":"2024-09-17T03:14:14.321286+0000","last_became_active":"2024-09-17T03:14:13.321392+0000","last_became_peered":"2024-09-17T03:14:13.321392+0000","last_unstale":"2024-09-17T03:14:14.321286+0000","last_undegraded":"2024-09-17T03:14:14.321286+0000","last_fullsized":"2024-09-17T03:14:14.321286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:33.594511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.346129+0000","last_change":"2024-09-17T03:14:14.341909+0000","last_active":"2024-09-17T03:14:14.346129+0000","last_peered":"2024-09-17T03:14:14.346129+0000","last_clean":"2024-09-17T03:14:14.346129+0000","last_became_active":"2024-09-17T03:14:14.341571+0000","last_became_peered":"2024-09-17T03:14:14.341571+0000","last_unstale":"2024-09-17T03:14:14.346129+0000","last_undegraded":"2024-09-17T03:14:14.346129+0000","last_fullsized":"2024-09-17T03:14:14.346129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:28.494598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.310378+0000","last_change":"2024-09-17T03:14:12.310378+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.310378+0000","last_undegraded":"2024-09-17T03:14:12.310378+0000","last_fullsized":"2024-09-17T03:14:12.310378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.344893+0000","last_change":"2024-09-17T03:14:14.340502+0000","last_active":"2024-09-17T03:14:14.344893+0000","last_peered":"2024-09-17T03:14:14.344893+0000","last_clean":"2024-09-17T03:14:14.344893+0000","last_became_active":"2024-09-17T03:14:14.340215+0000","last_became_peered":"2024-09-17T03:14:14.340215+0000","last_unstale":"2024-09-17T03:14:14.344893+0000","last_undegraded":"2024-09-17T03:14:14.344893+0000","last_fullsized":"2024-09-17T03:14:14.344893+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:46:40.874255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.308458+0000","last_change":"2024-09-17T03:14:12.308458+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.308458+0000","last_undegraded":"2024-09-17T03:14:12.308458+0000","last_fullsized":"2024-09-17T03:14:12.308458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.320414+0000","last_change":"2024-09-17T03:14:13.319558+0000","last_active":"2024-09-17T03:14:14.320414+0000","last_peered":"2024-09-17T03:14:14.320414+0000","last_clean":"2024-09-17T03:14:14.320414+0000","last_became_active":"2024-09-17T03:14:13.319236+0000","last_became_peered":"2024-09-17T03:14:13.319236+0000","last_unstale":"2024-09-17T03:14:14.320414+0000","last_undegraded":"2024-09-17T03:14:14.320414+0000","last_fullsized":"2024-09-17T03:14:14.320414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:25:39.260408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.337352+0000","last_change":"2024-09-17T03:14:14.329694+0000","last_active":"2024-09-17T03:14:14.337352+0000","last_peered":"2024-09-17T03:14:14.337352+0000","last_clean":"2024-09-17T03:14:14.337352+0000","last_became_active":"2024-09-17T03:14:14.329289+0000","last_became_peered":"2024-09-17T03:14:14.329289+0000","last_unstale":"2024-09-17T03:14:14.337352+0000","last_undegraded":"2024-09-17T03:14:14.337352+0000","last_fullsized":"2024-09-17T03:14:14.337352+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:55.345670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.309458+0000","last_change":"2024-09-17T03:14:12.309458+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.309458+0000","last_undegraded":"2024-09-17T03:14:12.309458+0000","last_fullsized":"2024-09-17T03:14:12.309458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.311838+0000","last_change":"2024-09-17T03:14:12.311838+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.311838+0000","last_undegraded":"2024-09-17T03:14:12.311838+0000","last_fullsized":"2024-09-17T03:14:12.311838+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.308581+0000","last_change":"2024-09-17T03:14:12.308581+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.308581+0000","last_undegraded":"2024-09-17T03:14:12.308581+0000","last_fullsized":"2024-09-17T03:14:12.308581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.308533+0000","last_change":"2024-09-17T03:14:12.308533+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.308533+0000","last_undegraded":"2024-09-17T03:14:12.308533+0000","last_fullsized":"2024-09-17T03:14:12.308533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.307080+0000","last_change":"2024-09-17T03:14:12.307080+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.307080+0000","last_undegraded":"2024-09-17T03:14:12.307080+0000","last_fullsized":"2024-09-17T03:14:12.307080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.351905+0000","last_change":"2024-09-17T03:14:13.324341+0000","last_active":"2024-09-17T03:14:14.351905+0000","last_peered":"2024-09-17T03:14:14.351905+0000","last_clean":"2024-09-17T03:14:14.351905+0000","last_became_active":"2024-09-17T03:14:13.323959+0000","last_became_peered":"2024-09-17T03:14:13.323959+0000","last_unstale":"2024-09-17T03:14:14.351905+0000","last_undegraded":"2024-09-17T03:14:14.351905+0000","last_fullsized":"2024-09-17T03:14:14.351905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:18:06.456762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14,"ondisk_log_size":14,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":56940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":184320,"data_stored":56940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":778240,"data_stored":647308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":196608,"data_stored":62757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":184320,"data_stored":56940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":778240,"data_stored":647308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":56940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":78091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:14:17.679 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:14:17.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.828+0000 7ff81f52b640 1 -- 172.21.15.87:0/3156748015 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff81815c560 msgr2=0x7ff818162d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:17.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.828+0000 7ff81f52b640 1 --2- 172.21.15.87:0/3156748015 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff81815c560 0x7ff818162d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff814005fd0 tx=0x7ff81402a5c0 comp rx=0 tx=0).stop 2024-09-17T03:14:17.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.828+0000 7ff81f52b640 1 -- 172.21.15.87:0/3156748015 shutdown_connections 2024-09-17T03:14:17.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.828+0000 7ff81f52b640 1 --2- 172.21.15.87:0/3156748015 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff8180b3d60 0x7ff8180b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:17.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.828+0000 7ff81f52b640 1 --2- 172.21.15.87:0/3156748015 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff81815c560 0x7ff818162d90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:17.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.828+0000 7ff81f52b640 1 -- 172.21.15.87:0/3156748015 >> 172.21.15.87:0/3156748015 conn(0x7ff818069330 msgr2=0x7ff8180ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:17.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.829+0000 7ff81f52b640 1 -- 172.21.15.87:0/3156748015 shutdown_connections 2024-09-17T03:14:17.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.829+0000 7ff81f52b640 1 -- 172.21.15.87:0/3156748015 wait complete. 2024-09-17T03:14:17.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.830+0000 7ff81f52b640 1 Processor -- start 2024-09-17T03:14:17.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.830+0000 7ff81f52b640 1 -- start start 2024-09-17T03:14:17.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.832+0000 7ff81f52b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8180710e0 0x7ff8180b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:17.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.832+0000 7ff81f52b640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff8180b3d60 0x7ff818151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:17.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.832+0000 7ff81f52b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff81815c560 0x7ff818160fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:17.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.832+0000 7ff81f52b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff81806f2c0 con 0x7ff8180710e0 2024-09-17T03:14:17.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.832+0000 7ff81f52b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff81806f140 con 0x7ff81815c560 2024-09-17T03:14:17.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.832+0000 7ff81f52b640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff81806f440 con 0x7ff8180b3d60 2024-09-17T03:14:17.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.832+0000 7ff81d2c7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8180710e0 0x7ff8180b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:17.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.832+0000 7ff81d2c7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8180710e0 0x7ff8180b7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39972/0 (socket says 172.21.15.87:39972) 2024-09-17T03:14:17.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.832+0000 7ff81d2c7640 1 -- 172.21.15.87:0/1349750920 learned_addr learned my addr 172.21.15.87:0/1349750920 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:17.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.832+0000 7ff81dac8640 1 --2- 172.21.15.87:0/1349750920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff81815c560 0x7ff818160fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:17.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.832+0000 7ff81d2c7640 1 -- 172.21.15.87:0/1349750920 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff8180b3d60 msgr2=0x7ff818151440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:17.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.832+0000 7ff81d2c7640 1 --2- 172.21.15.87:0/1349750920 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff8180b3d60 0x7ff818151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:17.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.832+0000 7ff81d2c7640 1 -- 172.21.15.87:0/1349750920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff81815c560 msgr2=0x7ff818160fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:17.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.832+0000 7ff81d2c7640 1 --2- 172.21.15.87:0/1349750920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff81815c560 0x7ff818160fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:17.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.832+0000 7ff81d2c7640 1 -- 172.21.15.87:0/1349750920 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8180b6160 con 0x7ff8180710e0 2024-09-17T03:14:17.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.832+0000 7ff81d2c7640 1 --2- 172.21.15.87:0/1349750920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8180710e0 0x7ff8180b7be0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff814034040 tx=0x7ff8140038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:17.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.832+0000 7ff80dffb640 1 -- 172.21.15.87:0/1349750920 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81403e030 con 0x7ff8180710e0 2024-09-17T03:14:17.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.832+0000 7ff80dffb640 1 -- 172.21.15.87:0/1349750920 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff814003d90 con 0x7ff8180710e0 2024-09-17T03:14:17.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.832+0000 7ff81f52b640 1 -- 172.21.15.87:0/1349750920 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8180b8120 con 0x7ff8180710e0 2024-09-17T03:14:17.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.832+0000 7ff80dffb640 1 -- 172.21.15.87:0/1349750920 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814035040 con 0x7ff8180710e0 2024-09-17T03:14:17.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.833+0000 7ff81f52b640 1 -- 172.21.15.87:0/1349750920 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8180b6700 con 0x7ff8180710e0 2024-09-17T03:14:17.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.834+0000 7ff81f52b640 1 -- 172.21.15.87:0/1349750920 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8180d0740 con 0x7ff8180710e0 2024-09-17T03:14:17.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.834+0000 7ff80dffb640 1 -- 172.21.15.87:0/1349750920 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff814041030 con 0x7ff8180710e0 2024-09-17T03:14:17.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.834+0000 7ff80dffb640 1 --2- 172.21.15.87:0/1349750920 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff7fc078bc0 0x7ff7fc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:17.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.835+0000 7ff80dffb640 1 -- 172.21.15.87:0/1349750920 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff8140bb890 con 0x7ff8180710e0 2024-09-17T03:14:17.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.835+0000 7ff81cac6640 1 --2- 172.21.15.87:0/1349750920 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff7fc078bc0 0x7ff7fc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:17.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.835+0000 7ff81cac6640 1 --2- 172.21.15.87:0/1349750920 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff7fc078bc0 0x7ff7fc07b0d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff8080058c0 tx=0x7ff808005850 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:17.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:17.839+0000 7ff80dffb640 1 -- 172.21.15.87:0/1349750920 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff8180d0740 con 0x7ff8180710e0 2024-09-17T03:14:18.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.011+0000 7ff81f52b640 1 -- 172.21.15.87:0/1349750920 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff81815d3d0 con 0x7ff7fc078bc0 2024-09-17T03:14:18.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.015+0000 7ff80dffb640 1 -- 172.21.15.87:0/1349750920 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241501 (secure 0 0 0) 0x7ff81815d3d0 con 0x7ff7fc078bc0 2024-09-17T03:14:18.017 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:18.018 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-09-17T03:14:18.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.018+0000 7ff81f52b640 1 -- 172.21.15.87:0/1349750920 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff7fc078bc0 msgr2=0x7ff7fc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:18.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.018+0000 7ff81f52b640 1 --2- 172.21.15.87:0/1349750920 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff7fc078bc0 0x7ff7fc07b0d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff8080058c0 tx=0x7ff808005850 comp rx=0 tx=0).stop 2024-09-17T03:14:18.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.018+0000 7ff81f52b640 1 -- 172.21.15.87:0/1349750920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8180710e0 msgr2=0x7ff8180b7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:18.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.018+0000 7ff81f52b640 1 --2- 172.21.15.87:0/1349750920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8180710e0 0x7ff8180b7be0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff814034040 tx=0x7ff8140038a0 comp rx=0 tx=0).stop 2024-09-17T03:14:18.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.018+0000 7ff81f52b640 1 -- 172.21.15.87:0/1349750920 shutdown_connections 2024-09-17T03:14:18.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.019+0000 7ff81f52b640 1 --2- 172.21.15.87:0/1349750920 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff7fc078bc0 0x7ff7fc07b0d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:18.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.019+0000 7ff81f52b640 1 --2- 172.21.15.87:0/1349750920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff81815c560 0x7ff818160fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:18.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.019+0000 7ff81f52b640 1 --2- 172.21.15.87:0/1349750920 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff8180b3d60 0x7ff818151440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:18.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.019+0000 7ff81f52b640 1 --2- 172.21.15.87:0/1349750920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8180710e0 0x7ff8180b7be0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:18.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.019+0000 7ff81f52b640 1 -- 172.21.15.87:0/1349750920 >> 172.21.15.87:0/1349750920 conn(0x7ff818069330 msgr2=0x7ff81815cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:18.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.019+0000 7ff81f52b640 1 -- 172.21.15.87:0/1349750920 shutdown_connections 2024-09-17T03:14:18.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.019+0000 7ff81f52b640 1 -- 172.21.15.87:0/1349750920 wait complete. 2024-09-17T03:14:18.028 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-17T03:14:17.417421+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":120,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":80,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219424,"kb_used_data":2668,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755296,"statfs":{"total":773094113280,"available":772869423104,"internally_reserved":0,"allocated":2732032,"data_stored":1663224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.114782"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.344245+0000","last_change":"2024-09-17T03:14:14.338758+0000","last_active":"2024-09-17T03:14:14.344245+0000","last_peered":"2024-09-17T03:14:14.344245+0000","last_clean":"2024-09-17T03:14:14.344245+0000","last_became_active":"2024-09-17T03:14:14.337821+0000","last_became_peered":"2024-09-17T03:14:14.337821+0000","last_unstale":"2024-09-17T03:14:14.344245+0000","last_undegraded":"2024-09-17T03:14:14.344245+0000","last_fullsized":"2024-09-17T03:14:14.344245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:47:29.951060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.307401+0000","last_change":"2024-09-17T03:14:12.307401+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.307401+0000","last_undegraded":"2024-09-17T03:14:12.307401+0000","last_fullsized":"2024-09-17T03:14:12.307401+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.338533+0000","last_change":"2024-09-17T03:14:14.330562+0000","last_active":"2024-09-17T03:14:14.338533+0000","last_peered":"2024-09-17T03:14:14.338533+0000","last_clean":"2024-09-17T03:14:14.338533+0000","last_became_active":"2024-09-17T03:14:14.330255+0000","last_became_peered":"2024-09-17T03:14:14.330255+0000","last_unstale":"2024-09-17T03:14:14.338533+0000","last_undegraded":"2024-09-17T03:14:14.338533+0000","last_fullsized":"2024-09-17T03:14:14.338533+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:17.454865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.311093+0000","last_change":"2024-09-17T03:14:12.311093+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.311093+0000","last_undegraded":"2024-09-17T03:14:12.311093+0000","last_fullsized":"2024-09-17T03:14:12.311093+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.309399+0000","last_change":"2024-09-17T03:14:12.309399+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.309399+0000","last_undegraded":"2024-09-17T03:14:12.309399+0000","last_fullsized":"2024-09-17T03:14:12.309399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.311069+0000","last_change":"2024-09-17T03:14:12.311069+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.311069+0000","last_undegraded":"2024-09-17T03:14:12.311069+0000","last_fullsized":"2024-09-17T03:14:12.311069+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.344304+0000","last_change":"2024-09-17T03:14:14.339215+0000","last_active":"2024-09-17T03:14:14.344304+0000","last_peered":"2024-09-17T03:14:14.344304+0000","last_clean":"2024-09-17T03:14:14.344304+0000","last_became_active":"2024-09-17T03:14:14.337738+0000","last_became_peered":"2024-09-17T03:14:14.337738+0000","last_unstale":"2024-09-17T03:14:14.344304+0000","last_undegraded":"2024-09-17T03:14:14.344304+0000","last_fullsized":"2024-09-17T03:14:14.344304+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:58.797420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.302832+0000","last_change":"2024-09-17T03:14:12.302832+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.302832+0000","last_undegraded":"2024-09-17T03:14:12.302832+0000","last_fullsized":"2024-09-17T03:14:12.302832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.308447+0000","last_change":"2024-09-17T03:14:12.308447+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.308447+0000","last_undegraded":"2024-09-17T03:14:12.308447+0000","last_fullsized":"2024-09-17T03:14:12.308447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.313764+0000","last_change":"2024-09-17T03:14:12.313764+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.313764+0000","last_undegraded":"2024-09-17T03:14:12.313764+0000","last_fullsized":"2024-09-17T03:14:12.313764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.311133+0000","last_change":"2024-09-17T03:14:12.311133+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.311133+0000","last_undegraded":"2024-09-17T03:14:12.311133+0000","last_fullsized":"2024-09-17T03:14:12.311133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.313748+0000","last_change":"2024-09-17T03:14:12.313748+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.313748+0000","last_undegraded":"2024-09-17T03:14:12.313748+0000","last_fullsized":"2024-09-17T03:14:12.313748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.310657+0000","last_change":"2024-09-17T03:14:12.310657+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.310657+0000","last_undegraded":"2024-09-17T03:14:12.310657+0000","last_fullsized":"2024-09-17T03:14:12.310657+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.308595+0000","last_change":"2024-09-17T03:14:12.308595+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.308595+0000","last_undegraded":"2024-09-17T03:14:12.308595+0000","last_fullsized":"2024-09-17T03:14:12.308595+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.345087+0000","last_change":"2024-09-17T03:14:13.321317+0000","last_active":"2024-09-17T03:14:14.345087+0000","last_peered":"2024-09-17T03:14:14.345087+0000","last_clean":"2024-09-17T03:14:14.345087+0000","last_became_active":"2024-09-17T03:14:13.320995+0000","last_became_peered":"2024-09-17T03:14:13.320995+0000","last_unstale":"2024-09-17T03:14:14.345087+0000","last_undegraded":"2024-09-17T03:14:14.345087+0000","last_fullsized":"2024-09-17T03:14:14.345087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:56.906875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.308531+0000","last_change":"2024-09-17T03:14:12.308531+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.308531+0000","last_undegraded":"2024-09-17T03:14:12.308531+0000","last_fullsized":"2024-09-17T03:14:12.308531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.345671+0000","last_change":"2024-09-17T03:14:13.322488+0000","last_active":"2024-09-17T03:14:14.345671+0000","last_peered":"2024-09-17T03:14:14.345671+0000","last_clean":"2024-09-17T03:14:14.345671+0000","last_became_active":"2024-09-17T03:14:13.322170+0000","last_became_peered":"2024-09-17T03:14:13.322170+0000","last_unstale":"2024-09-17T03:14:14.345671+0000","last_undegraded":"2024-09-17T03:14:14.345671+0000","last_fullsized":"2024-09-17T03:14:14.345671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:13.679632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.306379+0000","last_change":"2024-09-17T03:14:12.306379+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.306379+0000","last_undegraded":"2024-09-17T03:14:12.306379+0000","last_fullsized":"2024-09-17T03:14:12.306379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.309393+0000","last_change":"2024-09-17T03:14:12.309393+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.309393+0000","last_undegraded":"2024-09-17T03:14:12.309393+0000","last_fullsized":"2024-09-17T03:14:12.309393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":105,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T03:14:12.301021+0000","last_change":"2024-09-17T03:14:04.862167+0000","last_active":"2024-09-17T03:14:12.301021+0000","last_peered":"2024-09-17T03:14:12.301021+0000","last_clean":"2024-09-17T03:14:12.301021+0000","last_became_active":"2024-09-17T03:14:04.861731+0000","last_became_peered":"2024-09-17T03:14:04.861731+0000","last_unstale":"2024-09-17T03:14:12.301021+0000","last_undegraded":"2024-09-17T03:14:12.301021+0000","last_fullsized":"2024-09-17T03:14:12.301021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_clean_scrub_stamp":"2024-09-17T03:14:03.847721+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:32.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.311265+0000","last_change":"2024-09-17T03:14:12.311265+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.311265+0000","last_undegraded":"2024-09-17T03:14:12.311265+0000","last_fullsized":"2024-09-17T03:14:12.311265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.338243+0000","last_change":"2024-09-17T03:14:14.331745+0000","last_active":"2024-09-17T03:14:14.338243+0000","last_peered":"2024-09-17T03:14:14.338243+0000","last_clean":"2024-09-17T03:14:14.338243+0000","last_became_active":"2024-09-17T03:14:14.331207+0000","last_became_peered":"2024-09-17T03:14:14.331207+0000","last_unstale":"2024-09-17T03:14:14.338243+0000","last_undegraded":"2024-09-17T03:14:14.338243+0000","last_fullsized":"2024-09-17T03:14:14.338243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:32.588857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.309532+0000","last_change":"2024-09-17T03:14:12.309532+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.309532+0000","last_undegraded":"2024-09-17T03:14:12.309532+0000","last_fullsized":"2024-09-17T03:14:12.309532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.313614+0000","last_change":"2024-09-17T03:14:12.313614+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.313614+0000","last_undegraded":"2024-09-17T03:14:12.313614+0000","last_fullsized":"2024-09-17T03:14:12.313614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.310047+0000","last_change":"2024-09-17T03:14:12.310047+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.310047+0000","last_undegraded":"2024-09-17T03:14:12.310047+0000","last_fullsized":"2024-09-17T03:14:12.310047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.306474+0000","last_change":"2024-09-17T03:14:12.306474+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.306474+0000","last_undegraded":"2024-09-17T03:14:12.306474+0000","last_fullsized":"2024-09-17T03:14:12.306474+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.309954+0000","last_change":"2024-09-17T03:14:12.309954+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.309954+0000","last_undegraded":"2024-09-17T03:14:12.309954+0000","last_fullsized":"2024-09-17T03:14:12.309954+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.321286+0000","last_change":"2024-09-17T03:14:13.321756+0000","last_active":"2024-09-17T03:14:14.321286+0000","last_peered":"2024-09-17T03:14:14.321286+0000","last_clean":"2024-09-17T03:14:14.321286+0000","last_became_active":"2024-09-17T03:14:13.321392+0000","last_became_peered":"2024-09-17T03:14:13.321392+0000","last_unstale":"2024-09-17T03:14:14.321286+0000","last_undegraded":"2024-09-17T03:14:14.321286+0000","last_fullsized":"2024-09-17T03:14:14.321286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:33.594511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.346129+0000","last_change":"2024-09-17T03:14:14.341909+0000","last_active":"2024-09-17T03:14:14.346129+0000","last_peered":"2024-09-17T03:14:14.346129+0000","last_clean":"2024-09-17T03:14:14.346129+0000","last_became_active":"2024-09-17T03:14:14.341571+0000","last_became_peered":"2024-09-17T03:14:14.341571+0000","last_unstale":"2024-09-17T03:14:14.346129+0000","last_undegraded":"2024-09-17T03:14:14.346129+0000","last_fullsized":"2024-09-17T03:14:14.346129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:28.494598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.310378+0000","last_change":"2024-09-17T03:14:12.310378+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.310378+0000","last_undegraded":"2024-09-17T03:14:12.310378+0000","last_fullsized":"2024-09-17T03:14:12.310378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.344893+0000","last_change":"2024-09-17T03:14:14.340502+0000","last_active":"2024-09-17T03:14:14.344893+0000","last_peered":"2024-09-17T03:14:14.344893+0000","last_clean":"2024-09-17T03:14:14.344893+0000","last_became_active":"2024-09-17T03:14:14.340215+0000","last_became_peered":"2024-09-17T03:14:14.340215+0000","last_unstale":"2024-09-17T03:14:14.344893+0000","last_undegraded":"2024-09-17T03:14:14.344893+0000","last_fullsized":"2024-09-17T03:14:14.344893+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:46:40.874255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.308458+0000","last_change":"2024-09-17T03:14:12.308458+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.308458+0000","last_undegraded":"2024-09-17T03:14:12.308458+0000","last_fullsized":"2024-09-17T03:14:12.308458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.320414+0000","last_change":"2024-09-17T03:14:13.319558+0000","last_active":"2024-09-17T03:14:14.320414+0000","last_peered":"2024-09-17T03:14:14.320414+0000","last_clean":"2024-09-17T03:14:14.320414+0000","last_became_active":"2024-09-17T03:14:13.319236+0000","last_became_peered":"2024-09-17T03:14:13.319236+0000","last_unstale":"2024-09-17T03:14:14.320414+0000","last_undegraded":"2024-09-17T03:14:14.320414+0000","last_fullsized":"2024-09-17T03:14:14.320414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:25:39.260408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.337352+0000","last_change":"2024-09-17T03:14:14.329694+0000","last_active":"2024-09-17T03:14:14.337352+0000","last_peered":"2024-09-17T03:14:14.337352+0000","last_clean":"2024-09-17T03:14:14.337352+0000","last_became_active":"2024-09-17T03:14:14.329289+0000","last_became_peered":"2024-09-17T03:14:14.329289+0000","last_unstale":"2024-09-17T03:14:14.337352+0000","last_undegraded":"2024-09-17T03:14:14.337352+0000","last_fullsized":"2024-09-17T03:14:14.337352+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:55.345670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.309458+0000","last_change":"2024-09-17T03:14:12.309458+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.309458+0000","last_undegraded":"2024-09-17T03:14:12.309458+0000","last_fullsized":"2024-09-17T03:14:12.309458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.311838+0000","last_change":"2024-09-17T03:14:12.311838+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.311838+0000","last_undegraded":"2024-09-17T03:14:12.311838+0000","last_fullsized":"2024-09-17T03:14:12.311838+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.308581+0000","last_change":"2024-09-17T03:14:12.308581+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.308581+0000","last_undegraded":"2024-09-17T03:14:12.308581+0000","last_fullsized":"2024-09-17T03:14:12.308581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.308533+0000","last_change":"2024-09-17T03:14:12.308533+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.308533+0000","last_undegraded":"2024-09-17T03:14:12.308533+0000","last_fullsized":"2024-09-17T03:14:12.308533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.307080+0000","last_change":"2024-09-17T03:14:12.307080+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.307080+0000","last_undegraded":"2024-09-17T03:14:12.307080+0000","last_fullsized":"2024-09-17T03:14:12.307080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.351905+0000","last_change":"2024-09-17T03:14:13.324341+0000","last_active":"2024-09-17T03:14:14.351905+0000","last_peered":"2024-09-17T03:14:14.351905+0000","last_clean":"2024-09-17T03:14:14.351905+0000","last_became_active":"2024-09-17T03:14:13.323959+0000","last_became_peered":"2024-09-17T03:14:13.323959+0000","last_unstale":"2024-09-17T03:14:14.351905+0000","last_undegraded":"2024-09-17T03:14:14.351905+0000","last_fullsized":"2024-09-17T03:14:14.351905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:18:06.456762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14,"ondisk_log_size":14,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":56940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":184320,"data_stored":56940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":778240,"data_stored":647308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":196608,"data_stored":62757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":184320,"data_stored":56940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":778240,"data_stored":647308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":56940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":78091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:14:18.032 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3f is not active+clean 2024-09-17T03:14:18.032 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.032 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3e is not active+clean 2024-09-17T03:14:18.032 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.032 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3d is not active+clean 2024-09-17T03:14:18.033 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.033 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3c is not active+clean 2024-09-17T03:14:18.033 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.033 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3b is not active+clean 2024-09-17T03:14:18.033 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.033 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3a is not active+clean 2024-09-17T03:14:18.033 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.033 INFO:tasks.ceph.ceph_manager.ceph:PG 3.39 is not active+clean 2024-09-17T03:14:18.034 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.034 INFO:tasks.ceph.ceph_manager.ceph:PG 3.38 is not active+clean 2024-09-17T03:14:18.034 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.034 INFO:tasks.ceph.ceph_manager.ceph:PG 3.37 is not active+clean 2024-09-17T03:14:18.034 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.034 INFO:tasks.ceph.ceph_manager.ceph:PG 3.36 is not active+clean 2024-09-17T03:14:18.034 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.034 INFO:tasks.ceph.ceph_manager.ceph:PG 3.35 is not active+clean 2024-09-17T03:14:18.035 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.035 INFO:tasks.ceph.ceph_manager.ceph:PG 3.34 is not active+clean 2024-09-17T03:14:18.035 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.035 INFO:tasks.ceph.ceph_manager.ceph:PG 3.33 is not active+clean 2024-09-17T03:14:18.035 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.035 INFO:tasks.ceph.ceph_manager.ceph:PG 3.32 is not active+clean 2024-09-17T03:14:18.035 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.035 INFO:tasks.ceph.ceph_manager.ceph:PG 3.31 is not active+clean 2024-09-17T03:14:18.035 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.035 INFO:tasks.ceph.ceph_manager.ceph:PG 3.30 is not active+clean 2024-09-17T03:14:18.035 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.036 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2f is not active+clean 2024-09-17T03:14:18.036 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.036 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2e is not active+clean 2024-09-17T03:14:18.036 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.036 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2d is not active+clean 2024-09-17T03:14:18.036 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.036 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2c is not active+clean 2024-09-17T03:14:18.036 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.036 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2a is not active+clean 2024-09-17T03:14:18.036 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.036 INFO:tasks.ceph.ceph_manager.ceph:PG 3.29 is not active+clean 2024-09-17T03:14:18.037 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.037 INFO:tasks.ceph.ceph_manager.ceph:PG 3.28 is not active+clean 2024-09-17T03:14:18.037 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.037 INFO:tasks.ceph.ceph_manager.ceph:PG 3.27 is not active+clean 2024-09-17T03:14:18.037 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.037 INFO:tasks.ceph.ceph_manager.ceph:PG 3.26 is not active+clean 2024-09-17T03:14:18.037 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.037 INFO:tasks.ceph.ceph_manager.ceph:PG 3.25 is not active+clean 2024-09-17T03:14:18.037 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.037 INFO:tasks.ceph.ceph_manager.ceph:PG 3.24 is not active+clean 2024-09-17T03:14:18.037 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.038 INFO:tasks.ceph.ceph_manager.ceph:PG 3.23 is not active+clean 2024-09-17T03:14:18.038 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.038 INFO:tasks.ceph.ceph_manager.ceph:PG 3.22 is not active+clean 2024-09-17T03:14:18.038 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.038 INFO:tasks.ceph.ceph_manager.ceph:PG 3.21 is not active+clean 2024-09-17T03:14:18.038 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.038 INFO:tasks.ceph.ceph_manager.ceph:PG 3.20 is not active+clean 2024-09-17T03:14:18.038 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.038 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1a is not active+clean 2024-09-17T03:14:18.038 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.038 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1b is not active+clean 2024-09-17T03:14:18.039 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.307401+0000', 'last_change': '2024-09-17T03:14:12.307401+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.307401+0000', 'last_undegraded': '2024-09-17T03:14:12.307401+0000', 'last_fullsized': '2024-09-17T03:14:12.307401+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:14:18.039 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1a is not active+clean 2024-09-17T03:14:18.039 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.311093+0000', 'last_change': '2024-09-17T03:14:12.311093+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.311093+0000', 'last_undegraded': '2024-09-17T03:14:12.311093+0000', 'last_fullsized': '2024-09-17T03:14:12.311093+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T03:14:18.039 INFO:tasks.ceph.ceph_manager.ceph:PG 2.19 is not active+clean 2024-09-17T03:14:18.039 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.309399+0000', 'last_change': '2024-09-17T03:14:12.309399+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.309399+0000', 'last_undegraded': '2024-09-17T03:14:12.309399+0000', 'last_fullsized': '2024-09-17T03:14:12.309399+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T03:14:18.039 INFO:tasks.ceph.ceph_manager.ceph:PG 3.18 is not active+clean 2024-09-17T03:14:18.039 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.039 INFO:tasks.ceph.ceph_manager.ceph:PG 2.18 is not active+clean 2024-09-17T03:14:18.039 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.311069+0000', 'last_change': '2024-09-17T03:14:12.311069+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.311069+0000', 'last_undegraded': '2024-09-17T03:14:12.311069+0000', 'last_fullsized': '2024-09-17T03:14:12.311069+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T03:14:18.039 INFO:tasks.ceph.ceph_manager.ceph:PG 2.17 is not active+clean 2024-09-17T03:14:18.039 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.302832+0000', 'last_change': '2024-09-17T03:14:12.302832+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.302832+0000', 'last_undegraded': '2024-09-17T03:14:12.302832+0000', 'last_fullsized': '2024-09-17T03:14:12.302832+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T03:14:18.040 INFO:tasks.ceph.ceph_manager.ceph:PG 3.16 is not active+clean 2024-09-17T03:14:18.040 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.040 INFO:tasks.ceph.ceph_manager.ceph:PG 2.16 is not active+clean 2024-09-17T03:14:18.040 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.308447+0000', 'last_change': '2024-09-17T03:14:12.308447+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.308447+0000', 'last_undegraded': '2024-09-17T03:14:12.308447+0000', 'last_fullsized': '2024-09-17T03:14:12.308447+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T03:14:18.040 INFO:tasks.ceph.ceph_manager.ceph:PG 3.17 is not active+clean 2024-09-17T03:14:18.040 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.040 INFO:tasks.ceph.ceph_manager.ceph:PG 2.15 is not active+clean 2024-09-17T03:14:18.040 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.313764+0000', 'last_change': '2024-09-17T03:14:12.313764+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.313764+0000', 'last_undegraded': '2024-09-17T03:14:12.313764+0000', 'last_fullsized': '2024-09-17T03:14:12.313764+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T03:14:18.040 INFO:tasks.ceph.ceph_manager.ceph:PG 3.14 is not active+clean 2024-09-17T03:14:18.040 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.041 INFO:tasks.ceph.ceph_manager.ceph:PG 2.14 is not active+clean 2024-09-17T03:14:18.041 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.311133+0000', 'last_change': '2024-09-17T03:14:12.311133+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.311133+0000', 'last_undegraded': '2024-09-17T03:14:12.311133+0000', 'last_fullsized': '2024-09-17T03:14:12.311133+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:14:18.041 INFO:tasks.ceph.ceph_manager.ceph:PG 3.15 is not active+clean 2024-09-17T03:14:18.041 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.041 INFO:tasks.ceph.ceph_manager.ceph:PG 2.13 is not active+clean 2024-09-17T03:14:18.041 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.313748+0000', 'last_change': '2024-09-17T03:14:12.313748+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.313748+0000', 'last_undegraded': '2024-09-17T03:14:12.313748+0000', 'last_fullsized': '2024-09-17T03:14:12.313748+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T03:14:18.041 INFO:tasks.ceph.ceph_manager.ceph:PG 3.12 is not active+clean 2024-09-17T03:14:18.041 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.041 INFO:tasks.ceph.ceph_manager.ceph:PG 2.12 is not active+clean 2024-09-17T03:14:18.041 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.310657+0000', 'last_change': '2024-09-17T03:14:12.310657+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.310657+0000', 'last_undegraded': '2024-09-17T03:14:12.310657+0000', 'last_fullsized': '2024-09-17T03:14:12.310657+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T03:14:18.041 INFO:tasks.ceph.ceph_manager.ceph:PG 3.13 is not active+clean 2024-09-17T03:14:18.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.042 INFO:tasks.ceph.ceph_manager.ceph:PG 2.11 is not active+clean 2024-09-17T03:14:18.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.308595+0000', 'last_change': '2024-09-17T03:14:12.308595+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.308595+0000', 'last_undegraded': '2024-09-17T03:14:12.308595+0000', 'last_fullsized': '2024-09-17T03:14:12.308595+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T03:14:18.042 INFO:tasks.ceph.ceph_manager.ceph:PG 3.10 is not active+clean 2024-09-17T03:14:18.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.042 INFO:tasks.ceph.ceph_manager.ceph:PG 3.11 is not active+clean 2024-09-17T03:14:18.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.042 INFO:tasks.ceph.ceph_manager.ceph:PG 2.f is not active+clean 2024-09-17T03:14:18.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.308531+0000', 'last_change': '2024-09-17T03:14:12.308531+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.308531+0000', 'last_undegraded': '2024-09-17T03:14:12.308531+0000', 'last_fullsized': '2024-09-17T03:14:12.308531+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T03:14:18.042 INFO:tasks.ceph.ceph_manager.ceph:PG 3.e is not active+clean 2024-09-17T03:14:18.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.043 INFO:tasks.ceph.ceph_manager.ceph:PG 3.f is not active+clean 2024-09-17T03:14:18.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.043 INFO:tasks.ceph.ceph_manager.ceph:PG 2.d is not active+clean 2024-09-17T03:14:18.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.306379+0000', 'last_change': '2024-09-17T03:14:12.306379+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.306379+0000', 'last_undegraded': '2024-09-17T03:14:12.306379+0000', 'last_fullsized': '2024-09-17T03:14:12.306379+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:14:18.043 INFO:tasks.ceph.ceph_manager.ceph:PG 3.c is not active+clean 2024-09-17T03:14:18.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.043 INFO:tasks.ceph.ceph_manager.ceph:PG 2.c is not active+clean 2024-09-17T03:14:18.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.309393+0000', 'last_change': '2024-09-17T03:14:12.309393+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.309393+0000', 'last_undegraded': '2024-09-17T03:14:12.309393+0000', 'last_fullsized': '2024-09-17T03:14:12.309393+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T03:14:18.043 INFO:tasks.ceph.ceph_manager.ceph:PG 3.d is not active+clean 2024-09-17T03:14:18.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.043 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2024-09-17T03:14:18.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.311265+0000', 'last_change': '2024-09-17T03:14:12.311265+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.311265+0000', 'last_undegraded': '2024-09-17T03:14:12.311265+0000', 'last_fullsized': '2024-09-17T03:14:12.311265+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T03:14:18.044 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2024-09-17T03:14:18.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.309532+0000', 'last_change': '2024-09-17T03:14:12.309532+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.309532+0000', 'last_undegraded': '2024-09-17T03:14:12.309532+0000', 'last_fullsized': '2024-09-17T03:14:12.309532+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T03:14:18.044 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1 is not active+clean 2024-09-17T03:14:18.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.044 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2024-09-17T03:14:18.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.313614+0000', 'last_change': '2024-09-17T03:14:12.313614+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.313614+0000', 'last_undegraded': '2024-09-17T03:14:12.313614+0000', 'last_fullsized': '2024-09-17T03:14:12.313614+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T03:14:18.044 INFO:tasks.ceph.ceph_manager.ceph:PG 3.0 is not active+clean 2024-09-17T03:14:18.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.044 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2024-09-17T03:14:18.044 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.310047+0000', 'last_change': '2024-09-17T03:14:12.310047+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.310047+0000', 'last_undegraded': '2024-09-17T03:14:12.310047+0000', 'last_fullsized': '2024-09-17T03:14:12.310047+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T03:14:18.045 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3 is not active+clean 2024-09-17T03:14:18.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.045 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2024-09-17T03:14:18.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.306474+0000', 'last_change': '2024-09-17T03:14:12.306474+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.306474+0000', 'last_undegraded': '2024-09-17T03:14:12.306474+0000', 'last_fullsized': '2024-09-17T03:14:12.306474+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:14:18.045 INFO:tasks.ceph.ceph_manager.ceph:PG 3.5 is not active+clean 2024-09-17T03:14:18.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.045 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2024-09-17T03:14:18.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.309954+0000', 'last_change': '2024-09-17T03:14:12.309954+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.309954+0000', 'last_undegraded': '2024-09-17T03:14:12.309954+0000', 'last_fullsized': '2024-09-17T03:14:12.309954+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T03:14:18.045 INFO:tasks.ceph.ceph_manager.ceph:PG 3.4 is not active+clean 2024-09-17T03:14:18.045 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.045 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2024-09-17T03:14:18.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.310378+0000', 'last_change': '2024-09-17T03:14:12.310378+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.310378+0000', 'last_undegraded': '2024-09-17T03:14:12.310378+0000', 'last_fullsized': '2024-09-17T03:14:12.310378+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T03:14:18.046 INFO:tasks.ceph.ceph_manager.ceph:PG 2.8 is not active+clean 2024-09-17T03:14:18.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.308458+0000', 'last_change': '2024-09-17T03:14:12.308458+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.308458+0000', 'last_undegraded': '2024-09-17T03:14:12.308458+0000', 'last_fullsized': '2024-09-17T03:14:12.308458+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T03:14:18.046 INFO:tasks.ceph.ceph_manager.ceph:PG 3.9 is not active+clean 2024-09-17T03:14:18.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.046 INFO:tasks.ceph.ceph_manager.ceph:PG 2.a is not active+clean 2024-09-17T03:14:18.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.309458+0000', 'last_change': '2024-09-17T03:14:12.309458+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.309458+0000', 'last_undegraded': '2024-09-17T03:14:12.309458+0000', 'last_fullsized': '2024-09-17T03:14:12.309458+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T03:14:18.046 INFO:tasks.ceph.ceph_manager.ceph:PG 3.b is not active+clean 2024-09-17T03:14:18.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.046 INFO:tasks.ceph.ceph_manager.ceph:PG 2.b is not active+clean 2024-09-17T03:14:18.046 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.311838+0000', 'last_change': '2024-09-17T03:14:12.311838+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.311838+0000', 'last_undegraded': '2024-09-17T03:14:12.311838+0000', 'last_fullsized': '2024-09-17T03:14:12.311838+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T03:14:18.047 INFO:tasks.ceph.ceph_manager.ceph:PG 3.a is not active+clean 2024-09-17T03:14:18.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.047 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1d is not active+clean 2024-09-17T03:14:18.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.047 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1c is not active+clean 2024-09-17T03:14:18.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.308581+0000', 'last_change': '2024-09-17T03:14:12.308581+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.308581+0000', 'last_undegraded': '2024-09-17T03:14:12.308581+0000', 'last_fullsized': '2024-09-17T03:14:12.308581+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T03:14:18.047 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1c is not active+clean 2024-09-17T03:14:18.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.047 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1d is not active+clean 2024-09-17T03:14:18.047 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.308533+0000', 'last_change': '2024-09-17T03:14:12.308533+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.308533+0000', 'last_undegraded': '2024-09-17T03:14:12.308533+0000', 'last_fullsized': '2024-09-17T03:14:12.308533+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T03:14:18.047 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1f is not active+clean 2024-09-17T03:14:18.048 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.048 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1e is not active+clean 2024-09-17T03:14:18.048 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:12.307080+0000', 'last_change': '2024-09-17T03:14:12.307080+0000', 'last_active': '2024-09-17T03:14:12.294019+0000', 'last_peered': '2024-09-17T03:14:12.294019+0000', 'last_clean': '2024-09-17T03:14:12.294019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:12.307080+0000', 'last_undegraded': '2024-09-17T03:14:12.307080+0000', 'last_fullsized': '2024-09-17T03:14:12.307080+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:12.294019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:14:18.048 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1e is not active+clean 2024-09-17T03:14:18.048 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:13.298793+0000', 'last_change': '2024-09-17T03:14:13.298793+0000', 'last_active': '2024-09-17T03:14:13.298793+0000', 'last_peered': '2024-09-17T03:14:13.298793+0000', 'last_clean': '2024-09-17T03:14:13.298793+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:13.298793+0000', 'last_undegraded': '2024-09-17T03:14:13.298793+0000', 'last_fullsized': '2024-09-17T03:14:13.298793+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:13.298793+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:14:18.049 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T03:14:18.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.179+0000 7f5f510e6640 1 -- 172.21.15.87:0/2123527802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f4c09aa30 msgr2=0x7f5f4c09ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:18.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.179+0000 7f5f510e6640 1 --2- 172.21.15.87:0/2123527802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f4c09aa30 0x7f5f4c09ae40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009830 tx=0x7f5f4002a480 comp rx=0 tx=0).stop 2024-09-17T03:14:18.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.179+0000 7f5f510e6640 1 -- 172.21.15.87:0/2123527802 shutdown_connections 2024-09-17T03:14:18.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.179+0000 7f5f510e6640 1 --2- 172.21.15.87:0/2123527802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f4c09aa30 0x7f5f4c09ae40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:18.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.179+0000 7f5f510e6640 1 -- 172.21.15.87:0/2123527802 >> 172.21.15.87:0/2123527802 conn(0x7f5f4c0b3d30 msgr2=0x7f5f4c0b4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:18.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.179+0000 7f5f510e6640 1 -- 172.21.15.87:0/2123527802 shutdown_connections 2024-09-17T03:14:18.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.179+0000 7f5f510e6640 1 -- 172.21.15.87:0/2123527802 wait complete. 2024-09-17T03:14:18.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.181+0000 7f5f510e6640 1 Processor -- start 2024-09-17T03:14:18.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.181+0000 7f5f510e6640 1 -- start start 2024-09-17T03:14:18.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.183+0000 7f5f510e6640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f4c09aa30 0x7f5f4c1554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:18.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.183+0000 7f5f510e6640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5f4c0b5090 0x7f5f4c1f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:18.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.183+0000 7f5f510e6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f4c0ce0a0 0x7f5f4c1f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:18.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.183+0000 7f5f510e6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5f4c09b8f0 con 0x7f5f4c0ce0a0 2024-09-17T03:14:18.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.183+0000 7f5f510e6640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f5f4c09b770 con 0x7f5f4c09aa30 2024-09-17T03:14:18.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.183+0000 7f5f510e6640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f5f4c0b5bb0 con 0x7f5f4c0b5090 2024-09-17T03:14:18.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.183+0000 7f5f4b59e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f4c0ce0a0 0x7f5f4c1f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:18.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.183+0000 7f5f4b59e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f4c0ce0a0 0x7f5f4c1f4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39996/0 (socket says 172.21.15.87:39996) 2024-09-17T03:14:18.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.183+0000 7f5f4b59e640 1 -- 172.21.15.87:0/1045156234 learned_addr learned my addr 172.21.15.87:0/1045156234 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:18.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.183+0000 7f5f4ad9d640 1 --2- 172.21.15.87:0/1045156234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f4c09aa30 0x7f5f4c1554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:18.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.183+0000 7f5f4a59c640 1 --2- 172.21.15.87:0/1045156234 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5f4c0b5090 0x7f5f4c1f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:18.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.183+0000 7f5f4b59e640 1 -- 172.21.15.87:0/1045156234 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5f4c0b5090 msgr2=0x7f5f4c1f40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:18.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.183+0000 7f5f4b59e640 1 --2- 172.21.15.87:0/1045156234 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5f4c0b5090 0x7f5f4c1f40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:18.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.183+0000 7f5f4b59e640 1 -- 172.21.15.87:0/1045156234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f4c09aa30 msgr2=0x7f5f4c1554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:18.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.183+0000 7f5f4b59e640 1 --2- 172.21.15.87:0/1045156234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f4c09aa30 0x7f5f4c1554e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:18.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.183+0000 7f5f4b59e640 1 -- 172.21.15.87:0/1045156234 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f4c1f7f90 con 0x7f5f4c0ce0a0 2024-09-17T03:14:18.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.183+0000 7f5f4a59c640 1 --2- 172.21.15.87:0/1045156234 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5f4c0b5090 0x7f5f4c1f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:18.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.183+0000 7f5f4b59e640 1 --2- 172.21.15.87:0/1045156234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f4c0ce0a0 0x7f5f4c1f4600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400cab0 tx=0x7f5f4400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:18.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.183+0000 7f5f337fe640 1 -- 172.21.15.87:0/1045156234 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f44016020 con 0x7f5f4c0ce0a0 2024-09-17T03:14:18.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.183+0000 7f5f337fe640 1 -- 172.21.15.87:0/1045156234 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f4400ad20 con 0x7f5f4c0ce0a0 2024-09-17T03:14:18.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.183+0000 7f5f510e6640 1 -- 172.21.15.87:0/1045156234 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f4c156020 con 0x7f5f4c0ce0a0 2024-09-17T03:14:18.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.183+0000 7f5f337fe640 1 -- 172.21.15.87:0/1045156234 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f44012660 con 0x7f5f4c0ce0a0 2024-09-17T03:14:18.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.183+0000 7f5f510e6640 1 -- 172.21.15.87:0/1045156234 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f4c1f79d0 con 0x7f5f4c0ce0a0 2024-09-17T03:14:18.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.184+0000 7f5f510e6640 1 -- 172.21.15.87:0/1045156234 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f4c09ae40 con 0x7f5f4c0ce0a0 2024-09-17T03:14:18.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.185+0000 7f5f337fe640 1 -- 172.21.15.87:0/1045156234 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5f44019020 con 0x7f5f4c0ce0a0 2024-09-17T03:14:18.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.186+0000 7f5f337fe640 1 --2- 172.21.15.87:0/1045156234 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5f38078b90 0x7f5f3807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:18.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.186+0000 7f5f337fe640 1 -- 172.21.15.87:0/1045156234 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f5f4409af40 con 0x7f5f4c0ce0a0 2024-09-17T03:14:18.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.186+0000 7f5f4ad9d640 1 --2- 172.21.15.87:0/1045156234 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5f38078b90 0x7f5f3807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:18.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.186+0000 7f5f4ad9d640 1 --2- 172.21.15.87:0/1045156234 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5f38078b90 0x7f5f3807b0a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0098c0 tx=0x7f5f3c009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:18.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.190+0000 7f5f337fe640 1 -- 172.21.15.87:0/1045156234 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f5f44020030 con 0x7f5f4c0ce0a0 2024-09-17T03:14:18.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.431+0000 7f5f510e6640 1 -- 172.21.15.87:0/1045156234 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f5f4c0b4a70 con 0x7f5f4c0ce0a0 2024-09-17T03:14:18.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.431+0000 7f5f337fe640 1 -- 172.21.15.87:0/1045156234 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1213 (secure 0 0 0) 0x7f5f4c09b380 con 0x7f5f4c0ce0a0 2024-09-17T03:14:18.430 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:18.430 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":29,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":23,"num_osds":8,"num_up_osds":8,"osd_up_since":1726542841,"num_in_osds":8,"osd_in_since":1726542832,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":57},{"state_name":"creating+peering","count":27},{"state_name":"active+clean","count":13}],"num_pgs":97,"num_pools":3,"num_objects":9,"data_bytes":590950,"bytes_used":224690176,"bytes_avail":772869423104,"bytes_total":773094113280,"unknown_pgs_ratio":0.58762884140014648,"inactive_pgs_ratio":0.27835050225257874,"write_bytes_sec":200,"read_op_per_sec":0,"write_op_per_sec":3},"fsmap":{"epoch":6,"btime":"2024-09-17T03:14:16:330267+0000","id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"a","status":"up:active","gid":4398}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-17T03:13:48.955474+0000","services":{}},"progress_events":{}} 2024-09-17T03:14:18.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.433+0000 7f5f510e6640 1 -- 172.21.15.87:0/1045156234 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5f38078b90 msgr2=0x7f5f3807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:18.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.433+0000 7f5f510e6640 1 --2- 172.21.15.87:0/1045156234 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5f38078b90 0x7f5f3807b0a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0098c0 tx=0x7f5f3c009060 comp rx=0 tx=0).stop 2024-09-17T03:14:18.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.433+0000 7f5f510e6640 1 -- 172.21.15.87:0/1045156234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f4c0ce0a0 msgr2=0x7f5f4c1f4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:18.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.433+0000 7f5f510e6640 1 --2- 172.21.15.87:0/1045156234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f4c0ce0a0 0x7f5f4c1f4600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5f4400cab0 tx=0x7f5f4400cf70 comp rx=0 tx=0).stop 2024-09-17T03:14:18.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.433+0000 7f5f510e6640 1 -- 172.21.15.87:0/1045156234 shutdown_connections 2024-09-17T03:14:18.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.433+0000 7f5f510e6640 1 --2- 172.21.15.87:0/1045156234 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5f38078b90 0x7f5f3807b0a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:18.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.433+0000 7f5f510e6640 1 --2- 172.21.15.87:0/1045156234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f4c0ce0a0 0x7f5f4c1f4600 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:18.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.433+0000 7f5f510e6640 1 --2- 172.21.15.87:0/1045156234 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5f4c0b5090 0x7f5f4c1f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:18.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.433+0000 7f5f510e6640 1 --2- 172.21.15.87:0/1045156234 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f5f4c09aa30 0x7f5f4c1554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:18.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.433+0000 7f5f510e6640 1 -- 172.21.15.87:0/1045156234 >> 172.21.15.87:0/1045156234 conn(0x7f5f4c0b3d30 msgr2=0x7f5f4c0d08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:18.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.434+0000 7f5f510e6640 1 -- 172.21.15.87:0/1045156234 shutdown_connections 2024-09-17T03:14:18.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.434+0000 7f5f510e6640 1 -- 172.21.15.87:0/1045156234 wait complete. 2024-09-17T03:14:18.443 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T03:14:18.443 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:14:18.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.556+0000 7f66fee9b640 1 -- 172.21.15.87:0/3570180746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66f80b32a0 msgr2=0x7f66f80b36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:18.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.556+0000 7f66fee9b640 1 --2- 172.21.15.87:0/3570180746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66f80b32a0 0x7f66f80b36b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f66f80b3bf0 tx=0x7f66e002a560 comp rx=0 tx=0).stop 2024-09-17T03:14:18.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.556+0000 7f66fee9b640 1 -- 172.21.15.87:0/3570180746 shutdown_connections 2024-09-17T03:14:18.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.556+0000 7f66fee9b640 1 --2- 172.21.15.87:0/3570180746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66f80b32a0 0x7f66f80b36b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:18.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.556+0000 7f66fee9b640 1 --2- 172.21.15.87:0/3570180746 >> v2:172.21.15.149:3300/0 conn(0x7f66f80cfdf0 0x7f66f80b72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:18.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.556+0000 7f66fee9b640 1 -- 172.21.15.87:0/3570180746 >> 172.21.15.87:0/3570180746 conn(0x7f66f80b3d10 msgr2=0x7f66f80b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:18.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.556+0000 7f66fee9b640 1 -- 172.21.15.87:0/3570180746 shutdown_connections 2024-09-17T03:14:18.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.556+0000 7f66fee9b640 1 -- 172.21.15.87:0/3570180746 wait complete. 2024-09-17T03:14:18.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.558+0000 7f66fee9b640 1 Processor -- start 2024-09-17T03:14:18.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.558+0000 7f66fee9b640 1 -- start start 2024-09-17T03:14:18.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.559+0000 7f66fee9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66f80b32a0 0x7f66f815a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:18.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.559+0000 7f66fee9b640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f66f80cd990 0x7f66f80ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:18.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.559+0000 7f66fee9b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66f80cfdf0 0x7f66f80cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:18.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.559+0000 7f66fee9b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f66f80ce130 con 0x7f66f80b32a0 2024-09-17T03:14:18.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.559+0000 7f66fee9b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f66f80cdfb0 con 0x7f66f80cfdf0 2024-09-17T03:14:18.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.559+0000 7f66fee9b640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f66f80b2140 con 0x7f66f80cd990 2024-09-17T03:14:18.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.559+0000 7f66fcc37640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66f80b32a0 0x7f66f815a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:18.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.559+0000 7f66fcc37640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66f80b32a0 0x7f66f815a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40018/0 (socket says 172.21.15.87:40018) 2024-09-17T03:14:18.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.559+0000 7f66fcc37640 1 -- 172.21.15.87:0/3310892049 learned_addr learned my addr 172.21.15.87:0/3310892049 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:18.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.560+0000 7f66fd438640 1 --2- 172.21.15.87:0/3310892049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66f80cfdf0 0x7f66f80cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:18.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.560+0000 7f66f7fff640 1 --2- 172.21.15.87:0/3310892049 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f66f80cd990 0x7f66f80ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:18.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.560+0000 7f66fcc37640 1 -- 172.21.15.87:0/3310892049 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f66f80cd990 msgr2=0x7f66f80ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:18.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.560+0000 7f66fcc37640 1 --2- 172.21.15.87:0/3310892049 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f66f80cd990 0x7f66f80ca8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:18.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.560+0000 7f66fcc37640 1 -- 172.21.15.87:0/3310892049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66f80cfdf0 msgr2=0x7f66f80cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:18.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.560+0000 7f66fcc37640 1 --2- 172.21.15.87:0/3310892049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66f80cfdf0 0x7f66f80cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:18.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.560+0000 7f66fcc37640 1 -- 172.21.15.87:0/3310892049 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f815e510 con 0x7f66f80b32a0 2024-09-17T03:14:18.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.560+0000 7f66fd438640 1 --2- 172.21.15.87:0/3310892049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66f80cfdf0 0x7f66f80cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:18.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.560+0000 7f66fcc37640 1 --2- 172.21.15.87:0/3310892049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66f80b32a0 0x7f66f815a640 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f66e800b450 tx=0x7f66e800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:18.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.560+0000 7f66f57fa640 1 -- 172.21.15.87:0/3310892049 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e8015030 con 0x7f66f80b32a0 2024-09-17T03:14:18.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.560+0000 7f66f57fa640 1 -- 172.21.15.87:0/3310892049 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66e8005970 con 0x7f66f80b32a0 2024-09-17T03:14:18.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.560+0000 7f66f57fa640 1 -- 172.21.15.87:0/3310892049 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e8005dc0 con 0x7f66f80b32a0 2024-09-17T03:14:18.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.560+0000 7f66fee9b640 1 -- 172.21.15.87:0/3310892049 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f8160610 con 0x7f66f80b32a0 2024-09-17T03:14:18.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.560+0000 7f66fee9b640 1 -- 172.21.15.87:0/3310892049 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f806fbc0 con 0x7f66f80b32a0 2024-09-17T03:14:18.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.561+0000 7f66fee9b640 1 -- 172.21.15.87:0/3310892049 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66f80b36b0 con 0x7f66f80b32a0 2024-09-17T03:14:18.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.562+0000 7f66f57fa640 1 -- 172.21.15.87:0/3310892049 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f66e8018070 con 0x7f66f80b32a0 2024-09-17T03:14:18.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.562+0000 7f66f57fa640 1 --2- 172.21.15.87:0/3310892049 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f66d8078b90 0x7f66d807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:18.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.562+0000 7f66f57fa640 1 -- 172.21.15.87:0/3310892049 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f66e809a9b0 con 0x7f66f80b32a0 2024-09-17T03:14:18.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.563+0000 7f66f7fff640 1 --2- 172.21.15.87:0/3310892049 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f66d8078b90 0x7f66d807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:18.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.563+0000 7f66f7fff640 1 --2- 172.21.15.87:0/3310892049 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f66d8078b90 0x7f66d807b0a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f66e0033000 tx=0x7f66e0004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:18.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.567+0000 7f66f57fa640 1 -- 172.21.15.87:0/3310892049 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f66e800d040 con 0x7f66f80b32a0 2024-09-17T03:14:18.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.740+0000 7f66fee9b640 1 -- 172.21.15.87:0/3310892049 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f66f815acb0 con 0x7f66d8078b90 2024-09-17T03:14:18.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.744+0000 7f66f57fa640 1 -- 172.21.15.87:0/3310892049 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241501 (secure 0 0 0) 0x7f66f815acb0 con 0x7f66d8078b90 2024-09-17T03:14:18.745 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:18.746 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-09-17T03:14:18.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.747+0000 7f66fee9b640 1 -- 172.21.15.87:0/3310892049 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f66d8078b90 msgr2=0x7f66d807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:18.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.747+0000 7f66fee9b640 1 --2- 172.21.15.87:0/3310892049 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f66d8078b90 0x7f66d807b0a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f66e0033000 tx=0x7f66e0004730 comp rx=0 tx=0).stop 2024-09-17T03:14:18.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.748+0000 7f66fee9b640 1 -- 172.21.15.87:0/3310892049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66f80b32a0 msgr2=0x7f66f815a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:18.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.748+0000 7f66fee9b640 1 --2- 172.21.15.87:0/3310892049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66f80b32a0 0x7f66f815a640 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f66e800b450 tx=0x7f66e800b910 comp rx=0 tx=0).stop 2024-09-17T03:14:18.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.748+0000 7f66fee9b640 1 -- 172.21.15.87:0/3310892049 shutdown_connections 2024-09-17T03:14:18.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.748+0000 7f66fee9b640 1 --2- 172.21.15.87:0/3310892049 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f66d8078b90 0x7f66d807b0a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:18.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.748+0000 7f66fee9b640 1 --2- 172.21.15.87:0/3310892049 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66f80cfdf0 0x7f66f80cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:18.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.748+0000 7f66fee9b640 1 --2- 172.21.15.87:0/3310892049 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f66f80cd990 0x7f66f80ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:18.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.748+0000 7f66fee9b640 1 --2- 172.21.15.87:0/3310892049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66f80b32a0 0x7f66f815a640 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:18.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.748+0000 7f66fee9b640 1 -- 172.21.15.87:0/3310892049 >> 172.21.15.87:0/3310892049 conn(0x7f66f80b3d10 msgr2=0x7f66f81664f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:18.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.748+0000 7f66fee9b640 1 -- 172.21.15.87:0/3310892049 shutdown_connections 2024-09-17T03:14:18.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:18.748+0000 7f66fee9b640 1 -- 172.21.15.87:0/3310892049 wait complete. 2024-09-17T03:14:18.757 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-17T03:14:17.417421+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":120,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86,"ondisk_log_size":86,"up":80,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219424,"kb_used_data":2668,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755296,"statfs":{"total":773094113280,"available":772869423104,"internally_reserved":0,"allocated":2732032,"data_stored":1663224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.114782"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.344245+0000","last_change":"2024-09-17T03:14:14.338758+0000","last_active":"2024-09-17T03:14:14.344245+0000","last_peered":"2024-09-17T03:14:14.344245+0000","last_clean":"2024-09-17T03:14:14.344245+0000","last_became_active":"2024-09-17T03:14:14.337821+0000","last_became_peered":"2024-09-17T03:14:14.337821+0000","last_unstale":"2024-09-17T03:14:14.344245+0000","last_undegraded":"2024-09-17T03:14:14.344245+0000","last_fullsized":"2024-09-17T03:14:14.344245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:47:29.951060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.307401+0000","last_change":"2024-09-17T03:14:12.307401+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.307401+0000","last_undegraded":"2024-09-17T03:14:12.307401+0000","last_fullsized":"2024-09-17T03:14:12.307401+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.338533+0000","last_change":"2024-09-17T03:14:14.330562+0000","last_active":"2024-09-17T03:14:14.338533+0000","last_peered":"2024-09-17T03:14:14.338533+0000","last_clean":"2024-09-17T03:14:14.338533+0000","last_became_active":"2024-09-17T03:14:14.330255+0000","last_became_peered":"2024-09-17T03:14:14.330255+0000","last_unstale":"2024-09-17T03:14:14.338533+0000","last_undegraded":"2024-09-17T03:14:14.338533+0000","last_fullsized":"2024-09-17T03:14:14.338533+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:53:17.454865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.311093+0000","last_change":"2024-09-17T03:14:12.311093+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.311093+0000","last_undegraded":"2024-09-17T03:14:12.311093+0000","last_fullsized":"2024-09-17T03:14:12.311093+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.309399+0000","last_change":"2024-09-17T03:14:12.309399+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.309399+0000","last_undegraded":"2024-09-17T03:14:12.309399+0000","last_fullsized":"2024-09-17T03:14:12.309399+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.311069+0000","last_change":"2024-09-17T03:14:12.311069+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.311069+0000","last_undegraded":"2024-09-17T03:14:12.311069+0000","last_fullsized":"2024-09-17T03:14:12.311069+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.344304+0000","last_change":"2024-09-17T03:14:14.339215+0000","last_active":"2024-09-17T03:14:14.344304+0000","last_peered":"2024-09-17T03:14:14.344304+0000","last_clean":"2024-09-17T03:14:14.344304+0000","last_became_active":"2024-09-17T03:14:14.337738+0000","last_became_peered":"2024-09-17T03:14:14.337738+0000","last_unstale":"2024-09-17T03:14:14.344304+0000","last_undegraded":"2024-09-17T03:14:14.344304+0000","last_fullsized":"2024-09-17T03:14:14.344304+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:40:58.797420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.302832+0000","last_change":"2024-09-17T03:14:12.302832+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.302832+0000","last_undegraded":"2024-09-17T03:14:12.302832+0000","last_fullsized":"2024-09-17T03:14:12.302832+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.308447+0000","last_change":"2024-09-17T03:14:12.308447+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.308447+0000","last_undegraded":"2024-09-17T03:14:12.308447+0000","last_fullsized":"2024-09-17T03:14:12.308447+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.313764+0000","last_change":"2024-09-17T03:14:12.313764+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.313764+0000","last_undegraded":"2024-09-17T03:14:12.313764+0000","last_fullsized":"2024-09-17T03:14:12.313764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.311133+0000","last_change":"2024-09-17T03:14:12.311133+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.311133+0000","last_undegraded":"2024-09-17T03:14:12.311133+0000","last_fullsized":"2024-09-17T03:14:12.311133+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.313748+0000","last_change":"2024-09-17T03:14:12.313748+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.313748+0000","last_undegraded":"2024-09-17T03:14:12.313748+0000","last_fullsized":"2024-09-17T03:14:12.313748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.310657+0000","last_change":"2024-09-17T03:14:12.310657+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.310657+0000","last_undegraded":"2024-09-17T03:14:12.310657+0000","last_fullsized":"2024-09-17T03:14:12.310657+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.308595+0000","last_change":"2024-09-17T03:14:12.308595+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.308595+0000","last_undegraded":"2024-09-17T03:14:12.308595+0000","last_fullsized":"2024-09-17T03:14:12.308595+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.345087+0000","last_change":"2024-09-17T03:14:13.321317+0000","last_active":"2024-09-17T03:14:14.345087+0000","last_peered":"2024-09-17T03:14:14.345087+0000","last_clean":"2024-09-17T03:14:14.345087+0000","last_became_active":"2024-09-17T03:14:13.320995+0000","last_became_peered":"2024-09-17T03:14:13.320995+0000","last_unstale":"2024-09-17T03:14:14.345087+0000","last_undegraded":"2024-09-17T03:14:14.345087+0000","last_fullsized":"2024-09-17T03:14:14.345087+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:56.906875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.308531+0000","last_change":"2024-09-17T03:14:12.308531+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.308531+0000","last_undegraded":"2024-09-17T03:14:12.308531+0000","last_fullsized":"2024-09-17T03:14:12.308531+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.345671+0000","last_change":"2024-09-17T03:14:13.322488+0000","last_active":"2024-09-17T03:14:14.345671+0000","last_peered":"2024-09-17T03:14:14.345671+0000","last_clean":"2024-09-17T03:14:14.345671+0000","last_became_active":"2024-09-17T03:14:13.322170+0000","last_became_peered":"2024-09-17T03:14:13.322170+0000","last_unstale":"2024-09-17T03:14:14.345671+0000","last_undegraded":"2024-09-17T03:14:14.345671+0000","last_fullsized":"2024-09-17T03:14:14.345671+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:13.679632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.306379+0000","last_change":"2024-09-17T03:14:12.306379+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.306379+0000","last_undegraded":"2024-09-17T03:14:12.306379+0000","last_fullsized":"2024-09-17T03:14:12.306379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.309393+0000","last_change":"2024-09-17T03:14:12.309393+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.309393+0000","last_undegraded":"2024-09-17T03:14:12.309393+0000","last_fullsized":"2024-09-17T03:14:12.309393+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":105,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-17T03:14:12.301021+0000","last_change":"2024-09-17T03:14:04.862167+0000","last_active":"2024-09-17T03:14:12.301021+0000","last_peered":"2024-09-17T03:14:12.301021+0000","last_clean":"2024-09-17T03:14:12.301021+0000","last_became_active":"2024-09-17T03:14:04.861731+0000","last_became_peered":"2024-09-17T03:14:04.861731+0000","last_unstale":"2024-09-17T03:14:12.301021+0000","last_undegraded":"2024-09-17T03:14:12.301021+0000","last_fullsized":"2024-09-17T03:14:12.301021+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_clean_scrub_stamp":"2024-09-17T03:14:03.847721+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:32.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.311265+0000","last_change":"2024-09-17T03:14:12.311265+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.311265+0000","last_undegraded":"2024-09-17T03:14:12.311265+0000","last_fullsized":"2024-09-17T03:14:12.311265+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.338243+0000","last_change":"2024-09-17T03:14:14.331745+0000","last_active":"2024-09-17T03:14:14.338243+0000","last_peered":"2024-09-17T03:14:14.338243+0000","last_clean":"2024-09-17T03:14:14.338243+0000","last_became_active":"2024-09-17T03:14:14.331207+0000","last_became_peered":"2024-09-17T03:14:14.331207+0000","last_unstale":"2024-09-17T03:14:14.338243+0000","last_undegraded":"2024-09-17T03:14:14.338243+0000","last_fullsized":"2024-09-17T03:14:14.338243+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:32.588857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.309532+0000","last_change":"2024-09-17T03:14:12.309532+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.309532+0000","last_undegraded":"2024-09-17T03:14:12.309532+0000","last_fullsized":"2024-09-17T03:14:12.309532+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.313614+0000","last_change":"2024-09-17T03:14:12.313614+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.313614+0000","last_undegraded":"2024-09-17T03:14:12.313614+0000","last_fullsized":"2024-09-17T03:14:12.313614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.310047+0000","last_change":"2024-09-17T03:14:12.310047+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.310047+0000","last_undegraded":"2024-09-17T03:14:12.310047+0000","last_fullsized":"2024-09-17T03:14:12.310047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.306474+0000","last_change":"2024-09-17T03:14:12.306474+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.306474+0000","last_undegraded":"2024-09-17T03:14:12.306474+0000","last_fullsized":"2024-09-17T03:14:12.306474+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.309954+0000","last_change":"2024-09-17T03:14:12.309954+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.309954+0000","last_undegraded":"2024-09-17T03:14:12.309954+0000","last_fullsized":"2024-09-17T03:14:12.309954+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.321286+0000","last_change":"2024-09-17T03:14:13.321756+0000","last_active":"2024-09-17T03:14:14.321286+0000","last_peered":"2024-09-17T03:14:14.321286+0000","last_clean":"2024-09-17T03:14:14.321286+0000","last_became_active":"2024-09-17T03:14:13.321392+0000","last_became_peered":"2024-09-17T03:14:13.321392+0000","last_unstale":"2024-09-17T03:14:14.321286+0000","last_undegraded":"2024-09-17T03:14:14.321286+0000","last_fullsized":"2024-09-17T03:14:14.321286+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:33.594511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.346129+0000","last_change":"2024-09-17T03:14:14.341909+0000","last_active":"2024-09-17T03:14:14.346129+0000","last_peered":"2024-09-17T03:14:14.346129+0000","last_clean":"2024-09-17T03:14:14.346129+0000","last_became_active":"2024-09-17T03:14:14.341571+0000","last_became_peered":"2024-09-17T03:14:14.341571+0000","last_unstale":"2024-09-17T03:14:14.346129+0000","last_undegraded":"2024-09-17T03:14:14.346129+0000","last_fullsized":"2024-09-17T03:14:14.346129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:53:28.494598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.310378+0000","last_change":"2024-09-17T03:14:12.310378+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.310378+0000","last_undegraded":"2024-09-17T03:14:12.310378+0000","last_fullsized":"2024-09-17T03:14:12.310378+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.344893+0000","last_change":"2024-09-17T03:14:14.340502+0000","last_active":"2024-09-17T03:14:14.344893+0000","last_peered":"2024-09-17T03:14:14.344893+0000","last_clean":"2024-09-17T03:14:14.344893+0000","last_became_active":"2024-09-17T03:14:14.340215+0000","last_became_peered":"2024-09-17T03:14:14.340215+0000","last_unstale":"2024-09-17T03:14:14.344893+0000","last_undegraded":"2024-09-17T03:14:14.344893+0000","last_fullsized":"2024-09-17T03:14:14.344893+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:46:40.874255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.308458+0000","last_change":"2024-09-17T03:14:12.308458+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.308458+0000","last_undegraded":"2024-09-17T03:14:12.308458+0000","last_fullsized":"2024-09-17T03:14:12.308458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.320414+0000","last_change":"2024-09-17T03:14:13.319558+0000","last_active":"2024-09-17T03:14:14.320414+0000","last_peered":"2024-09-17T03:14:14.320414+0000","last_clean":"2024-09-17T03:14:14.320414+0000","last_became_active":"2024-09-17T03:14:13.319236+0000","last_became_peered":"2024-09-17T03:14:13.319236+0000","last_unstale":"2024-09-17T03:14:14.320414+0000","last_undegraded":"2024-09-17T03:14:14.320414+0000","last_fullsized":"2024-09-17T03:14:14.320414+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:25:39.260408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.337352+0000","last_change":"2024-09-17T03:14:14.329694+0000","last_active":"2024-09-17T03:14:14.337352+0000","last_peered":"2024-09-17T03:14:14.337352+0000","last_clean":"2024-09-17T03:14:14.337352+0000","last_became_active":"2024-09-17T03:14:14.329289+0000","last_became_peered":"2024-09-17T03:14:14.329289+0000","last_unstale":"2024-09-17T03:14:14.337352+0000","last_undegraded":"2024-09-17T03:14:14.337352+0000","last_fullsized":"2024-09-17T03:14:14.337352+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:26:55.345670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.309458+0000","last_change":"2024-09-17T03:14:12.309458+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.309458+0000","last_undegraded":"2024-09-17T03:14:12.309458+0000","last_fullsized":"2024-09-17T03:14:12.309458+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.311838+0000","last_change":"2024-09-17T03:14:12.311838+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.311838+0000","last_undegraded":"2024-09-17T03:14:12.311838+0000","last_fullsized":"2024-09-17T03:14:12.311838+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.308581+0000","last_change":"2024-09-17T03:14:12.308581+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.308581+0000","last_undegraded":"2024-09-17T03:14:12.308581+0000","last_fullsized":"2024-09-17T03:14:12.308581+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.308533+0000","last_change":"2024-09-17T03:14:12.308533+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.308533+0000","last_undegraded":"2024-09-17T03:14:12.308533+0000","last_fullsized":"2024-09-17T03:14:12.308533+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-17T03:14:12.307080+0000","last_change":"2024-09-17T03:14:12.307080+0000","last_active":"2024-09-17T03:14:12.294019+0000","last_peered":"2024-09-17T03:14:12.294019+0000","last_clean":"2024-09-17T03:14:12.294019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:12.307080+0000","last_undegraded":"2024-09-17T03:14:12.307080+0000","last_fullsized":"2024-09-17T03:14:12.307080+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:13.298793+0000","last_change":"2024-09-17T03:14:13.298793+0000","last_active":"2024-09-17T03:14:13.298793+0000","last_peered":"2024-09-17T03:14:13.298793+0000","last_clean":"2024-09-17T03:14:13.298793+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:13.298793+0000","last_undegraded":"2024-09-17T03:14:13.298793+0000","last_fullsized":"2024-09-17T03:14:13.298793+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-17T03:14:14.351905+0000","last_change":"2024-09-17T03:14:13.324341+0000","last_active":"2024-09-17T03:14:14.351905+0000","last_peered":"2024-09-17T03:14:14.351905+0000","last_clean":"2024-09-17T03:14:14.351905+0000","last_became_active":"2024-09-17T03:14:13.323959+0000","last_became_peered":"2024-09-17T03:14:13.323959+0000","last_unstale":"2024-09-17T03:14:14.351905+0000","last_undegraded":"2024-09-17T03:14:14.351905+0000","last_fullsized":"2024-09-17T03:14:14.351905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:18:06.456762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14,"ondisk_log_size":14,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":56940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":184320,"data_stored":56940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":778240,"data_stored":647308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":196608,"data_stored":62757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509446,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":184320,"data_stored":56940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509446,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28036,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343804,"statfs":{"total":96636764160,"available":96608055296,"internally_reserved":0,"allocated":778240,"data_stored":647308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":184320,"data_stored":56940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":78091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:14:19.289 INFO:tasks.ceph.mds.a.smithi087.stderr:2024-09-17T03:14:19.290+0000 7f5ce803d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T03:14:21.762 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:14:21.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.889+0000 7ffbbb189640 1 -- 172.21.15.87:0/2220198826 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffbb40b3d00 msgr2=0x7ffbb40b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:21.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.889+0000 7ffbbb189640 1 --2- 172.21.15.87:0/2220198826 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffbb40b3d00 0x7ffbb40b4110 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffba0009830 tx=0x7ffba002a480 comp rx=0 tx=0).stop 2024-09-17T03:14:21.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.889+0000 7ffbbb189640 1 -- 172.21.15.87:0/2220198826 shutdown_connections 2024-09-17T03:14:21.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.889+0000 7ffbbb189640 1 --2- 172.21.15.87:0/2220198826 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffbb40b3d00 0x7ffbb40b4110 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:21.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.889+0000 7ffbbb189640 1 --2- 172.21.15.87:0/2220198826 >> v2:172.21.15.149:3300/0 conn(0x7ffbb415c6c0 0x7ffbb4160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:21.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.889+0000 7ffbbb189640 1 -- 172.21.15.87:0/2220198826 >> 172.21.15.87:0/2220198826 conn(0x7ffbb4069360 msgr2=0x7ffbb40ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:21.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.889+0000 7ffbbb189640 1 -- 172.21.15.87:0/2220198826 shutdown_connections 2024-09-17T03:14:21.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.889+0000 7ffbbb189640 1 -- 172.21.15.87:0/2220198826 wait complete. 2024-09-17T03:14:21.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.890+0000 7ffbbb189640 1 Processor -- start 2024-09-17T03:14:21.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.891+0000 7ffbbb189640 1 -- start start 2024-09-17T03:14:21.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.892+0000 7ffbbb189640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffbb40710e0 0x7ffbb4148ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:21.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.892+0000 7ffbbb189640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffbb40b3d00 0x7ffbb415f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:21.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.892+0000 7ffbbb189640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffbb415c6c0 0x7ffbb41f3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:21.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.892+0000 7ffbbb189640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb40d1470 con 0x7ffbb40b3d00 2024-09-17T03:14:21.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.892+0000 7ffbbb189640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffbb40b80d0 con 0x7ffbb40710e0 2024-09-17T03:14:21.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.892+0000 7ffbbb189640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ffbb40d15f0 con 0x7ffbb415c6c0 2024-09-17T03:14:21.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.892+0000 7ffbb3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffbb40b3d00 0x7ffbb415f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:21.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.892+0000 7ffbb3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffbb40b3d00 0x7ffbb415f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40026/0 (socket says 172.21.15.87:40026) 2024-09-17T03:14:21.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.892+0000 7ffbb3fff640 1 -- 172.21.15.87:0/1601064745 learned_addr learned my addr 172.21.15.87:0/1601064745 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:21.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.892+0000 7ffbb9726640 1 --2- 172.21.15.87:0/1601064745 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffbb415c6c0 0x7ffbb41f3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:21.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.892+0000 7ffbb8f25640 1 --2- 172.21.15.87:0/1601064745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffbb40710e0 0x7ffbb4148ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:21.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.892+0000 7ffbb3fff640 1 -- 172.21.15.87:0/1601064745 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffbb415c6c0 msgr2=0x7ffbb41f3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:21.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.893+0000 7ffbb3fff640 1 --2- 172.21.15.87:0/1601064745 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffbb415c6c0 0x7ffbb41f3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:21.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.893+0000 7ffbb3fff640 1 -- 172.21.15.87:0/1601064745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffbb40710e0 msgr2=0x7ffbb4148ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:21.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.893+0000 7ffbb3fff640 1 --2- 172.21.15.87:0/1601064745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffbb40710e0 0x7ffbb4148ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:21.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.893+0000 7ffbb3fff640 1 -- 172.21.15.87:0/1601064745 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbb40cad20 con 0x7ffbb40b3d00 2024-09-17T03:14:21.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.893+0000 7ffbb9726640 1 --2- 172.21.15.87:0/1601064745 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffbb415c6c0 0x7ffbb41f3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:21.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.893+0000 7ffbb3fff640 1 --2- 172.21.15.87:0/1601064745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffbb40b3d00 0x7ffbb415f5e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffba002aa70 tx=0x7ffba0033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:21.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.893+0000 7ffbb17fa640 1 -- 172.21.15.87:0/1601064745 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba0031d60 con 0x7ffbb40b3d00 2024-09-17T03:14:21.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.893+0000 7ffbb17fa640 1 -- 172.21.15.87:0/1601064745 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffba0033e10 con 0x7ffbb40b3d00 2024-09-17T03:14:21.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.893+0000 7ffbbb189640 1 -- 172.21.15.87:0/1601064745 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbb415d1c0 con 0x7ffbb40b3d00 2024-09-17T03:14:21.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.893+0000 7ffbb17fa640 1 -- 172.21.15.87:0/1601064745 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffba0005260 con 0x7ffbb40b3d00 2024-09-17T03:14:21.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.893+0000 7ffbbb189640 1 -- 172.21.15.87:0/1601064745 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbb40cb290 con 0x7ffbb40b3d00 2024-09-17T03:14:21.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.894+0000 7ffbbb189640 1 -- 172.21.15.87:0/1601064745 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbb415cf30 con 0x7ffbb40b3d00 2024-09-17T03:14:21.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.895+0000 7ffbb17fa640 1 -- 172.21.15.87:0/1601064745 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffba0005400 con 0x7ffbb40b3d00 2024-09-17T03:14:21.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.895+0000 7ffbb17fa640 1 --2- 172.21.15.87:0/1601064745 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ffb98078b90 0x7ffb9807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:21.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.895+0000 7ffbb17fa640 1 -- 172.21.15.87:0/1601064745 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ffba00bbe70 con 0x7ffbb40b3d00 2024-09-17T03:14:21.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.896+0000 7ffbb8f25640 1 --2- 172.21.15.87:0/1601064745 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ffb98078b90 0x7ffb9807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:21.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.896+0000 7ffbb8f25640 1 --2- 172.21.15.87:0/1601064745 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ffb98078b90 0x7ffb9807b0a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ffba8005fd0 tx=0x7ffba8009260 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:21.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:21.900+0000 7ffbb17fa640 1 -- 172.21.15.87:0/1601064745 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ffba003f030 con 0x7ffbb40b3d00 2024-09-17T03:14:22.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.070+0000 7ffbbb189640 1 -- 172.21.15.87:0/1601064745 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffbb4071ce0 con 0x7ffb98078b90 2024-09-17T03:14:22.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.075+0000 7ffbb17fa640 1 -- 172.21.15.87:0/1601064745 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251921 (secure 0 0 0) 0x7ffbb4071ce0 con 0x7ffb98078b90 2024-09-17T03:14:22.076 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:22.077 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-09-17T03:14:22.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.078+0000 7ffbbb189640 1 -- 172.21.15.87:0/1601064745 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ffb98078b90 msgr2=0x7ffb9807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:22.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.078+0000 7ffbbb189640 1 --2- 172.21.15.87:0/1601064745 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ffb98078b90 0x7ffb9807b0a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ffba8005fd0 tx=0x7ffba8009260 comp rx=0 tx=0).stop 2024-09-17T03:14:22.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.078+0000 7ffbbb189640 1 -- 172.21.15.87:0/1601064745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffbb40b3d00 msgr2=0x7ffbb415f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:22.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.078+0000 7ffbbb189640 1 --2- 172.21.15.87:0/1601064745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffbb40b3d00 0x7ffbb415f5e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffba002aa70 tx=0x7ffba0033c20 comp rx=0 tx=0).stop 2024-09-17T03:14:22.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.078+0000 7ffbbb189640 1 -- 172.21.15.87:0/1601064745 shutdown_connections 2024-09-17T03:14:22.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.078+0000 7ffbbb189640 1 --2- 172.21.15.87:0/1601064745 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ffb98078b90 0x7ffb9807b0a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.078+0000 7ffbbb189640 1 --2- 172.21.15.87:0/1601064745 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffbb415c6c0 0x7ffbb41f3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.078+0000 7ffbbb189640 1 --2- 172.21.15.87:0/1601064745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffbb40b3d00 0x7ffbb415f5e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.078+0000 7ffbbb189640 1 --2- 172.21.15.87:0/1601064745 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffbb40710e0 0x7ffbb4148ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.078+0000 7ffbbb189640 1 -- 172.21.15.87:0/1601064745 >> 172.21.15.87:0/1601064745 conn(0x7ffbb4069360 msgr2=0x7ffbb40c85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:22.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.078+0000 7ffbbb189640 1 -- 172.21.15.87:0/1601064745 shutdown_connections 2024-09-17T03:14:22.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.078+0000 7ffbbb189640 1 -- 172.21.15.87:0/1601064745 wait complete. 2024-09-17T03:14:22.087 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-17T03:14:21.418305+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219280,"kb_used_data":3048,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755440,"statfs":{"total":773094113280,"available":772869570560,"internally_reserved":0,"allocated":3121152,"data_stored":1842412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001841"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332877+0000","last_change":"2024-09-17T03:14:14.350186+0000","last_active":"2024-09-17T03:14:15.332877+0000","last_peered":"2024-09-17T03:14:15.332877+0000","last_clean":"2024-09-17T03:14:15.332877+0000","last_became_active":"2024-09-17T03:14:14.349855+0000","last_became_peered":"2024-09-17T03:14:14.349855+0000","last_unstale":"2024-09-17T03:14:15.332877+0000","last_undegraded":"2024-09-17T03:14:15.332877+0000","last_fullsized":"2024-09-17T03:14:15.332877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:56:09.730649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334764+0000","last_change":"2024-09-17T03:14:14.343284+0000","last_active":"2024-09-17T03:14:15.334764+0000","last_peered":"2024-09-17T03:14:15.334764+0000","last_clean":"2024-09-17T03:14:15.334764+0000","last_became_active":"2024-09-17T03:14:14.337583+0000","last_became_peered":"2024-09-17T03:14:14.337583+0000","last_unstale":"2024-09-17T03:14:15.334764+0000","last_undegraded":"2024-09-17T03:14:15.334764+0000","last_fullsized":"2024-09-17T03:14:15.334764+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:06:49.022656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334676+0000","last_change":"2024-09-17T03:14:14.340288+0000","last_active":"2024-09-17T03:14:15.334676+0000","last_peered":"2024-09-17T03:14:15.334676+0000","last_clean":"2024-09-17T03:14:15.334676+0000","last_became_active":"2024-09-17T03:14:14.338799+0000","last_became_peered":"2024-09-17T03:14:14.338799+0000","last_unstale":"2024-09-17T03:14:15.334676+0000","last_undegraded":"2024-09-17T03:14:15.334676+0000","last_fullsized":"2024-09-17T03:14:15.334676+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:16:54.254992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391881+0000","last_change":"2024-09-17T03:14:14.333722+0000","last_active":"2024-09-17T03:14:16.391881+0000","last_peered":"2024-09-17T03:14:16.391881+0000","last_clean":"2024-09-17T03:14:16.391881+0000","last_became_active":"2024-09-17T03:14:14.333401+0000","last_became_peered":"2024-09-17T03:14:14.333401+0000","last_unstale":"2024-09-17T03:14:16.391881+0000","last_undegraded":"2024-09-17T03:14:16.391881+0000","last_fullsized":"2024-09-17T03:14:16.391881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:40.499497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333292+0000","last_change":"2024-09-17T03:14:14.334841+0000","last_active":"2024-09-17T03:14:15.333292+0000","last_peered":"2024-09-17T03:14:15.333292+0000","last_clean":"2024-09-17T03:14:15.333292+0000","last_became_active":"2024-09-17T03:14:14.334543+0000","last_became_peered":"2024-09-17T03:14:14.334543+0000","last_unstale":"2024-09-17T03:14:15.333292+0000","last_undegraded":"2024-09-17T03:14:15.333292+0000","last_fullsized":"2024-09-17T03:14:15.333292+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:51.158396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391311+0000","last_change":"2024-09-17T03:14:14.339302+0000","last_active":"2024-09-17T03:14:16.391311+0000","last_peered":"2024-09-17T03:14:16.391311+0000","last_clean":"2024-09-17T03:14:16.391311+0000","last_became_active":"2024-09-17T03:14:14.331863+0000","last_became_peered":"2024-09-17T03:14:14.331863+0000","last_unstale":"2024-09-17T03:14:16.391311+0000","last_undegraded":"2024-09-17T03:14:16.391311+0000","last_fullsized":"2024-09-17T03:14:16.391311+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:40.639682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332440+0000","last_change":"2024-09-17T03:14:14.340253+0000","last_active":"2024-09-17T03:14:15.332440+0000","last_peered":"2024-09-17T03:14:15.332440+0000","last_clean":"2024-09-17T03:14:15.332440+0000","last_became_active":"2024-09-17T03:14:14.339855+0000","last_became_peered":"2024-09-17T03:14:14.339855+0000","last_unstale":"2024-09-17T03:14:15.332440+0000","last_undegraded":"2024-09-17T03:14:15.332440+0000","last_fullsized":"2024-09-17T03:14:15.332440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:44:20.684218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334380+0000","last_change":"2024-09-17T03:14:14.333479+0000","last_active":"2024-09-17T03:14:15.334380+0000","last_peered":"2024-09-17T03:14:15.334380+0000","last_clean":"2024-09-17T03:14:15.334380+0000","last_became_active":"2024-09-17T03:14:14.330382+0000","last_became_peered":"2024-09-17T03:14:14.330382+0000","last_unstale":"2024-09-17T03:14:15.334380+0000","last_undegraded":"2024-09-17T03:14:15.334380+0000","last_fullsized":"2024-09-17T03:14:15.334380+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:21.969469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391159+0000","last_change":"2024-09-17T03:14:14.345634+0000","last_active":"2024-09-17T03:14:16.391159+0000","last_peered":"2024-09-17T03:14:16.391159+0000","last_clean":"2024-09-17T03:14:16.391159+0000","last_became_active":"2024-09-17T03:14:14.345309+0000","last_became_peered":"2024-09-17T03:14:14.345309+0000","last_unstale":"2024-09-17T03:14:16.391159+0000","last_undegraded":"2024-09-17T03:14:16.391159+0000","last_fullsized":"2024-09-17T03:14:16.391159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:09.963317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332555+0000","last_change":"2024-09-17T03:14:14.337246+0000","last_active":"2024-09-17T03:14:15.332555+0000","last_peered":"2024-09-17T03:14:15.332555+0000","last_clean":"2024-09-17T03:14:15.332555+0000","last_became_active":"2024-09-17T03:14:14.336920+0000","last_became_peered":"2024-09-17T03:14:14.336920+0000","last_unstale":"2024-09-17T03:14:15.332555+0000","last_undegraded":"2024-09-17T03:14:15.332555+0000","last_fullsized":"2024-09-17T03:14:15.332555+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:35.974141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332769+0000","last_change":"2024-09-17T03:14:14.349173+0000","last_active":"2024-09-17T03:14:15.332769+0000","last_peered":"2024-09-17T03:14:15.332769+0000","last_clean":"2024-09-17T03:14:15.332769+0000","last_became_active":"2024-09-17T03:14:14.348420+0000","last_became_peered":"2024-09-17T03:14:14.348420+0000","last_unstale":"2024-09-17T03:14:15.332769+0000","last_undegraded":"2024-09-17T03:14:15.332769+0000","last_fullsized":"2024-09-17T03:14:15.332769+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:09.339640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332966+0000","last_change":"2024-09-17T03:14:14.336719+0000","last_active":"2024-09-17T03:14:15.332966+0000","last_peered":"2024-09-17T03:14:15.332966+0000","last_clean":"2024-09-17T03:14:15.332966+0000","last_became_active":"2024-09-17T03:14:14.336239+0000","last_became_peered":"2024-09-17T03:14:14.336239+0000","last_unstale":"2024-09-17T03:14:15.332966+0000","last_undegraded":"2024-09-17T03:14:15.332966+0000","last_fullsized":"2024-09-17T03:14:15.332966+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:15:59.839348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391861+0000","last_change":"2024-09-17T03:14:14.340337+0000","last_active":"2024-09-17T03:14:16.391861+0000","last_peered":"2024-09-17T03:14:16.391861+0000","last_clean":"2024-09-17T03:14:16.391861+0000","last_became_active":"2024-09-17T03:14:14.340028+0000","last_became_peered":"2024-09-17T03:14:14.340028+0000","last_unstale":"2024-09-17T03:14:16.391861+0000","last_undegraded":"2024-09-17T03:14:16.391861+0000","last_fullsized":"2024-09-17T03:14:16.391861+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:35.806642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392067+0000","last_change":"2024-09-17T03:14:14.334276+0000","last_active":"2024-09-17T03:14:16.392067+0000","last_peered":"2024-09-17T03:14:16.392067+0000","last_clean":"2024-09-17T03:14:16.392067+0000","last_became_active":"2024-09-17T03:14:14.333764+0000","last_became_peered":"2024-09-17T03:14:14.333764+0000","last_unstale":"2024-09-17T03:14:16.392067+0000","last_undegraded":"2024-09-17T03:14:16.392067+0000","last_fullsized":"2024-09-17T03:14:16.392067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:57.801941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333421+0000","last_change":"2024-09-17T03:14:14.337055+0000","last_active":"2024-09-17T03:14:15.333421+0000","last_peered":"2024-09-17T03:14:15.333421+0000","last_clean":"2024-09-17T03:14:15.333421+0000","last_became_active":"2024-09-17T03:14:14.334416+0000","last_became_peered":"2024-09-17T03:14:14.334416+0000","last_unstale":"2024-09-17T03:14:15.333421+0000","last_undegraded":"2024-09-17T03:14:15.333421+0000","last_fullsized":"2024-09-17T03:14:15.333421+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:03.846825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332440+0000","last_change":"2024-09-17T03:14:14.346619+0000","last_active":"2024-09-17T03:14:15.332440+0000","last_peered":"2024-09-17T03:14:15.332440+0000","last_clean":"2024-09-17T03:14:15.332440+0000","last_became_active":"2024-09-17T03:14:14.346322+0000","last_became_peered":"2024-09-17T03:14:14.346322+0000","last_unstale":"2024-09-17T03:14:15.332440+0000","last_undegraded":"2024-09-17T03:14:15.332440+0000","last_fullsized":"2024-09-17T03:14:15.332440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:34:00.384055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334241+0000","last_change":"2024-09-17T03:14:14.343978+0000","last_active":"2024-09-17T03:14:15.334241+0000","last_peered":"2024-09-17T03:14:15.334241+0000","last_clean":"2024-09-17T03:14:15.334241+0000","last_became_active":"2024-09-17T03:14:14.343600+0000","last_became_peered":"2024-09-17T03:14:14.343600+0000","last_unstale":"2024-09-17T03:14:15.334241+0000","last_undegraded":"2024-09-17T03:14:15.334241+0000","last_fullsized":"2024-09-17T03:14:15.334241+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:25.368172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391936+0000","last_change":"2024-09-17T03:14:14.336744+0000","last_active":"2024-09-17T03:14:16.391936+0000","last_peered":"2024-09-17T03:14:16.391936+0000","last_clean":"2024-09-17T03:14:16.391936+0000","last_became_active":"2024-09-17T03:14:14.336428+0000","last_became_peered":"2024-09-17T03:14:14.336428+0000","last_unstale":"2024-09-17T03:14:16.391936+0000","last_undegraded":"2024-09-17T03:14:16.391936+0000","last_fullsized":"2024-09-17T03:14:16.391936+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:09.963317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334349+0000","last_change":"2024-09-17T03:14:14.348250+0000","last_active":"2024-09-17T03:14:15.334349+0000","last_peered":"2024-09-17T03:14:15.334349+0000","last_clean":"2024-09-17T03:14:15.334349+0000","last_became_active":"2024-09-17T03:14:14.347538+0000","last_became_peered":"2024-09-17T03:14:14.347538+0000","last_unstale":"2024-09-17T03:14:15.334349+0000","last_undegraded":"2024-09-17T03:14:15.334349+0000","last_fullsized":"2024-09-17T03:14:15.334349+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:46.734553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333193+0000","last_change":"2024-09-17T03:14:14.335802+0000","last_active":"2024-09-17T03:14:15.333193+0000","last_peered":"2024-09-17T03:14:15.333193+0000","last_clean":"2024-09-17T03:14:15.333193+0000","last_became_active":"2024-09-17T03:14:14.335507+0000","last_became_peered":"2024-09-17T03:14:14.335507+0000","last_unstale":"2024-09-17T03:14:15.333193+0000","last_undegraded":"2024-09-17T03:14:15.333193+0000","last_fullsized":"2024-09-17T03:14:15.333193+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:29.705538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333725+0000","last_change":"2024-09-17T03:14:14.338758+0000","last_active":"2024-09-17T03:14:15.333725+0000","last_peered":"2024-09-17T03:14:15.333725+0000","last_clean":"2024-09-17T03:14:15.333725+0000","last_became_active":"2024-09-17T03:14:14.337821+0000","last_became_peered":"2024-09-17T03:14:14.337821+0000","last_unstale":"2024-09-17T03:14:15.333725+0000","last_undegraded":"2024-09-17T03:14:15.333725+0000","last_fullsized":"2024-09-17T03:14:15.333725+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:09:24.347980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391355+0000","last_change":"2024-09-17T03:14:14.332615+0000","last_active":"2024-09-17T03:14:16.391355+0000","last_peered":"2024-09-17T03:14:16.391355+0000","last_clean":"2024-09-17T03:14:16.391355+0000","last_became_active":"2024-09-17T03:14:14.332285+0000","last_became_peered":"2024-09-17T03:14:14.332285+0000","last_unstale":"2024-09-17T03:14:16.391355+0000","last_undegraded":"2024-09-17T03:14:16.391355+0000","last_fullsized":"2024-09-17T03:14:16.391355+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:40.435687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333687+0000","last_change":"2024-09-17T03:14:14.342718+0000","last_active":"2024-09-17T03:14:15.333687+0000","last_peered":"2024-09-17T03:14:15.333687+0000","last_clean":"2024-09-17T03:14:15.333687+0000","last_became_active":"2024-09-17T03:14:14.341029+0000","last_became_peered":"2024-09-17T03:14:14.341029+0000","last_unstale":"2024-09-17T03:14:15.333687+0000","last_undegraded":"2024-09-17T03:14:15.333687+0000","last_fullsized":"2024-09-17T03:14:15.333687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:24:30.789181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332649+0000","last_change":"2024-09-17T03:14:14.344909+0000","last_active":"2024-09-17T03:14:15.332649+0000","last_peered":"2024-09-17T03:14:15.332649+0000","last_clean":"2024-09-17T03:14:15.332649+0000","last_became_active":"2024-09-17T03:14:14.343198+0000","last_became_peered":"2024-09-17T03:14:14.343198+0000","last_unstale":"2024-09-17T03:14:15.332649+0000","last_undegraded":"2024-09-17T03:14:15.332649+0000","last_fullsized":"2024-09-17T03:14:15.332649+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:06.251650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334992+0000","last_change":"2024-09-17T03:14:14.339599+0000","last_active":"2024-09-17T03:14:15.334992+0000","last_peered":"2024-09-17T03:14:15.334992+0000","last_clean":"2024-09-17T03:14:15.334992+0000","last_became_active":"2024-09-17T03:14:14.339248+0000","last_became_peered":"2024-09-17T03:14:14.339248+0000","last_unstale":"2024-09-17T03:14:15.334992+0000","last_undegraded":"2024-09-17T03:14:15.334992+0000","last_fullsized":"2024-09-17T03:14:15.334992+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:57:00.810832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392086+0000","last_change":"2024-09-17T03:14:14.347769+0000","last_active":"2024-09-17T03:14:16.392086+0000","last_peered":"2024-09-17T03:14:16.392086+0000","last_clean":"2024-09-17T03:14:16.392086+0000","last_became_active":"2024-09-17T03:14:14.347449+0000","last_became_peered":"2024-09-17T03:14:14.347449+0000","last_unstale":"2024-09-17T03:14:16.392086+0000","last_undegraded":"2024-09-17T03:14:16.392086+0000","last_fullsized":"2024-09-17T03:14:16.392086+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:14.698488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392056+0000","last_change":"2024-09-17T03:14:14.334291+0000","last_active":"2024-09-17T03:14:16.392056+0000","last_peered":"2024-09-17T03:14:16.392056+0000","last_clean":"2024-09-17T03:14:16.392056+0000","last_became_active":"2024-09-17T03:14:14.333993+0000","last_became_peered":"2024-09-17T03:14:14.333993+0000","last_unstale":"2024-09-17T03:14:16.392056+0000","last_undegraded":"2024-09-17T03:14:16.392056+0000","last_fullsized":"2024-09-17T03:14:16.392056+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:57.558924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392260+0000","last_change":"2024-09-17T03:14:14.344498+0000","last_active":"2024-09-17T03:14:16.392260+0000","last_peered":"2024-09-17T03:14:16.392260+0000","last_clean":"2024-09-17T03:14:16.392260+0000","last_became_active":"2024-09-17T03:14:14.344151+0000","last_became_peered":"2024-09-17T03:14:14.344151+0000","last_unstale":"2024-09-17T03:14:16.392260+0000","last_undegraded":"2024-09-17T03:14:16.392260+0000","last_fullsized":"2024-09-17T03:14:16.392260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:04:24.782583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334896+0000","last_change":"2024-09-17T03:14:14.327110+0000","last_active":"2024-09-17T03:14:15.334896+0000","last_peered":"2024-09-17T03:14:15.334896+0000","last_clean":"2024-09-17T03:14:15.334896+0000","last_became_active":"2024-09-17T03:14:14.326521+0000","last_became_peered":"2024-09-17T03:14:14.326521+0000","last_unstale":"2024-09-17T03:14:15.334896+0000","last_undegraded":"2024-09-17T03:14:15.334896+0000","last_fullsized":"2024-09-17T03:14:15.334896+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:42.804342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333824+0000","last_change":"2024-09-17T03:14:14.344297+0000","last_active":"2024-09-17T03:14:15.333824+0000","last_peered":"2024-09-17T03:14:15.333824+0000","last_clean":"2024-09-17T03:14:15.333824+0000","last_became_active":"2024-09-17T03:14:14.343941+0000","last_became_peered":"2024-09-17T03:14:14.343941+0000","last_unstale":"2024-09-17T03:14:15.333824+0000","last_undegraded":"2024-09-17T03:14:15.333824+0000","last_fullsized":"2024-09-17T03:14:15.333824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:18:47.095575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391581+0000","last_change":"2024-09-17T03:14:14.335330+0000","last_active":"2024-09-17T03:14:16.391581+0000","last_peered":"2024-09-17T03:14:16.391581+0000","last_clean":"2024-09-17T03:14:16.391581+0000","last_became_active":"2024-09-17T03:14:14.334989+0000","last_became_peered":"2024-09-17T03:14:14.334989+0000","last_unstale":"2024-09-17T03:14:16.391581+0000","last_undegraded":"2024-09-17T03:14:16.391581+0000","last_fullsized":"2024-09-17T03:14:16.391581+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:30.375807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391256+0000","last_change":"2024-09-17T03:14:14.334915+0000","last_active":"2024-09-17T03:14:16.391256+0000","last_peered":"2024-09-17T03:14:16.391256+0000","last_clean":"2024-09-17T03:14:16.391256+0000","last_became_active":"2024-09-17T03:14:14.334587+0000","last_became_peered":"2024-09-17T03:14:14.334587+0000","last_unstale":"2024-09-17T03:14:16.391256+0000","last_undegraded":"2024-09-17T03:14:16.391256+0000","last_fullsized":"2024-09-17T03:14:16.391256+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:55:20.376892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391571+0000","last_change":"2024-09-17T03:14:14.335030+0000","last_active":"2024-09-17T03:14:16.391571+0000","last_peered":"2024-09-17T03:14:16.391571+0000","last_clean":"2024-09-17T03:14:16.391571+0000","last_became_active":"2024-09-17T03:14:14.334722+0000","last_became_peered":"2024-09-17T03:14:14.334722+0000","last_unstale":"2024-09-17T03:14:16.391571+0000","last_undegraded":"2024-09-17T03:14:16.391571+0000","last_fullsized":"2024-09-17T03:14:16.391571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:59:38.941073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334406+0000","last_change":"2024-09-17T03:14:13.319793+0000","last_active":"2024-09-17T03:14:15.334406+0000","last_peered":"2024-09-17T03:14:15.334406+0000","last_clean":"2024-09-17T03:14:15.334406+0000","last_became_active":"2024-09-17T03:14:13.319467+0000","last_became_peered":"2024-09-17T03:14:13.319467+0000","last_unstale":"2024-09-17T03:14:15.334406+0000","last_undegraded":"2024-09-17T03:14:15.334406+0000","last_fullsized":"2024-09-17T03:14:15.334406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:06:54.724381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333960+0000","last_change":"2024-09-17T03:14:14.330562+0000","last_active":"2024-09-17T03:14:15.333960+0000","last_peered":"2024-09-17T03:14:15.333960+0000","last_clean":"2024-09-17T03:14:15.333960+0000","last_became_active":"2024-09-17T03:14:14.330255+0000","last_became_peered":"2024-09-17T03:14:14.330255+0000","last_unstale":"2024-09-17T03:14:15.333960+0000","last_undegraded":"2024-09-17T03:14:15.333960+0000","last_fullsized":"2024-09-17T03:14:15.333960+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:52:15.666625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333906+0000","last_change":"2024-09-17T03:14:13.324436+0000","last_active":"2024-09-17T03:14:15.333906+0000","last_peered":"2024-09-17T03:14:15.333906+0000","last_clean":"2024-09-17T03:14:15.333906+0000","last_became_active":"2024-09-17T03:14:13.323350+0000","last_became_peered":"2024-09-17T03:14:13.323350+0000","last_unstale":"2024-09-17T03:14:15.333906+0000","last_undegraded":"2024-09-17T03:14:15.333906+0000","last_fullsized":"2024-09-17T03:14:15.333906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:13.645071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391387+0000","last_change":"2024-09-17T03:14:13.330532+0000","last_active":"2024-09-17T03:14:16.391387+0000","last_peered":"2024-09-17T03:14:16.391387+0000","last_clean":"2024-09-17T03:14:16.391387+0000","last_became_active":"2024-09-17T03:14:13.330217+0000","last_became_peered":"2024-09-17T03:14:13.330217+0000","last_unstale":"2024-09-17T03:14:16.391387+0000","last_undegraded":"2024-09-17T03:14:16.391387+0000","last_fullsized":"2024-09-17T03:14:16.391387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:00.266996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334660+0000","last_change":"2024-09-17T03:14:14.337981+0000","last_active":"2024-09-17T03:14:15.334660+0000","last_peered":"2024-09-17T03:14:15.334660+0000","last_clean":"2024-09-17T03:14:15.334660+0000","last_became_active":"2024-09-17T03:14:14.336212+0000","last_became_peered":"2024-09-17T03:14:14.336212+0000","last_unstale":"2024-09-17T03:14:15.334660+0000","last_undegraded":"2024-09-17T03:14:15.334660+0000","last_fullsized":"2024-09-17T03:14:15.334660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:12:16.212384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332704+0000","last_change":"2024-09-17T03:14:13.316453+0000","last_active":"2024-09-17T03:14:15.332704+0000","last_peered":"2024-09-17T03:14:15.332704+0000","last_clean":"2024-09-17T03:14:15.332704+0000","last_became_active":"2024-09-17T03:14:13.316149+0000","last_became_peered":"2024-09-17T03:14:13.316149+0000","last_unstale":"2024-09-17T03:14:15.332704+0000","last_undegraded":"2024-09-17T03:14:15.332704+0000","last_fullsized":"2024-09-17T03:14:15.332704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:39.494722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334061+0000","last_change":"2024-09-17T03:14:14.339215+0000","last_active":"2024-09-17T03:14:15.334061+0000","last_peered":"2024-09-17T03:14:15.334061+0000","last_clean":"2024-09-17T03:14:15.334061+0000","last_became_active":"2024-09-17T03:14:14.337738+0000","last_became_peered":"2024-09-17T03:14:14.337738+0000","last_unstale":"2024-09-17T03:14:15.334061+0000","last_undegraded":"2024-09-17T03:14:15.334061+0000","last_fullsized":"2024-09-17T03:14:15.334061+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:18.278040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391218+0000","last_change":"2024-09-17T03:14:13.321965+0000","last_active":"2024-09-17T03:14:16.391218+0000","last_peered":"2024-09-17T03:14:16.391218+0000","last_clean":"2024-09-17T03:14:16.391218+0000","last_became_active":"2024-09-17T03:14:13.321616+0000","last_became_peered":"2024-09-17T03:14:13.321616+0000","last_unstale":"2024-09-17T03:14:16.391218+0000","last_undegraded":"2024-09-17T03:14:16.391218+0000","last_fullsized":"2024-09-17T03:14:16.391218+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:05:46.263336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392183+0000","last_change":"2024-09-17T03:14:14.348468+0000","last_active":"2024-09-17T03:14:16.392183+0000","last_peered":"2024-09-17T03:14:16.392183+0000","last_clean":"2024-09-17T03:14:16.392183+0000","last_became_active":"2024-09-17T03:14:14.348144+0000","last_became_peered":"2024-09-17T03:14:14.348144+0000","last_unstale":"2024-09-17T03:14:16.392183+0000","last_undegraded":"2024-09-17T03:14:16.392183+0000","last_fullsized":"2024-09-17T03:14:16.392183+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:14.649846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332338+0000","last_change":"2024-09-17T03:14:13.329841+0000","last_active":"2024-09-17T03:14:15.332338+0000","last_peered":"2024-09-17T03:14:15.332338+0000","last_clean":"2024-09-17T03:14:15.332338+0000","last_became_active":"2024-09-17T03:14:13.329154+0000","last_became_peered":"2024-09-17T03:14:13.329154+0000","last_unstale":"2024-09-17T03:14:15.332338+0000","last_undegraded":"2024-09-17T03:14:15.332338+0000","last_fullsized":"2024-09-17T03:14:15.332338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:39.634907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.335175+0000","last_change":"2024-09-17T03:14:14.345392+0000","last_active":"2024-09-17T03:14:15.335175+0000","last_peered":"2024-09-17T03:14:15.335175+0000","last_clean":"2024-09-17T03:14:15.335175+0000","last_became_active":"2024-09-17T03:14:14.345073+0000","last_became_peered":"2024-09-17T03:14:14.345073+0000","last_unstale":"2024-09-17T03:14:15.335175+0000","last_undegraded":"2024-09-17T03:14:15.335175+0000","last_fullsized":"2024-09-17T03:14:15.335175+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:04.528125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391685+0000","last_change":"2024-09-17T03:14:13.313377+0000","last_active":"2024-09-17T03:14:16.391685+0000","last_peered":"2024-09-17T03:14:16.391685+0000","last_clean":"2024-09-17T03:14:16.391685+0000","last_became_active":"2024-09-17T03:14:13.312848+0000","last_became_peered":"2024-09-17T03:14:13.312848+0000","last_unstale":"2024-09-17T03:14:16.391685+0000","last_undegraded":"2024-09-17T03:14:16.391685+0000","last_fullsized":"2024-09-17T03:14:16.391685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:00.266996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333208+0000","last_change":"2024-09-17T03:14:14.344920+0000","last_active":"2024-09-17T03:14:15.333208+0000","last_peered":"2024-09-17T03:14:15.333208+0000","last_clean":"2024-09-17T03:14:15.333208+0000","last_became_active":"2024-09-17T03:14:14.344570+0000","last_became_peered":"2024-09-17T03:14:14.344570+0000","last_unstale":"2024-09-17T03:14:15.333208+0000","last_undegraded":"2024-09-17T03:14:15.333208+0000","last_fullsized":"2024-09-17T03:14:15.333208+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:03.086199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391710+0000","last_change":"2024-09-17T03:14:13.322066+0000","last_active":"2024-09-17T03:14:16.391710+0000","last_peered":"2024-09-17T03:14:16.391710+0000","last_clean":"2024-09-17T03:14:16.391710+0000","last_became_active":"2024-09-17T03:14:13.321679+0000","last_became_peered":"2024-09-17T03:14:13.321679+0000","last_unstale":"2024-09-17T03:14:16.391710+0000","last_undegraded":"2024-09-17T03:14:16.391710+0000","last_fullsized":"2024-09-17T03:14:16.391710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:55.304571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333004+0000","last_change":"2024-09-17T03:14:14.337601+0000","last_active":"2024-09-17T03:14:15.333004+0000","last_peered":"2024-09-17T03:14:15.333004+0000","last_clean":"2024-09-17T03:14:15.333004+0000","last_became_active":"2024-09-17T03:14:14.336457+0000","last_became_peered":"2024-09-17T03:14:14.336457+0000","last_unstale":"2024-09-17T03:14:15.333004+0000","last_undegraded":"2024-09-17T03:14:15.333004+0000","last_fullsized":"2024-09-17T03:14:15.333004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:53:37.876227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391686+0000","last_change":"2024-09-17T03:14:13.318626+0000","last_active":"2024-09-17T03:14:16.391686+0000","last_peered":"2024-09-17T03:14:16.391686+0000","last_clean":"2024-09-17T03:14:16.391686+0000","last_became_active":"2024-09-17T03:14:13.318302+0000","last_became_peered":"2024-09-17T03:14:13.318302+0000","last_unstale":"2024-09-17T03:14:16.391686+0000","last_undegraded":"2024-09-17T03:14:16.391686+0000","last_fullsized":"2024-09-17T03:14:16.391686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:53.797340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392030+0000","last_change":"2024-09-17T03:14:14.344640+0000","last_active":"2024-09-17T03:14:16.392030+0000","last_peered":"2024-09-17T03:14:16.392030+0000","last_clean":"2024-09-17T03:14:16.392030+0000","last_became_active":"2024-09-17T03:14:14.344288+0000","last_became_peered":"2024-09-17T03:14:14.344288+0000","last_unstale":"2024-09-17T03:14:16.392030+0000","last_undegraded":"2024-09-17T03:14:16.392030+0000","last_fullsized":"2024-09-17T03:14:16.392030+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:55:42.826482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333916+0000","last_change":"2024-09-17T03:14:13.317055+0000","last_active":"2024-09-17T03:14:15.333916+0000","last_peered":"2024-09-17T03:14:15.333916+0000","last_clean":"2024-09-17T03:14:15.333916+0000","last_became_active":"2024-09-17T03:14:13.316736+0000","last_became_peered":"2024-09-17T03:14:13.316736+0000","last_unstale":"2024-09-17T03:14:15.333916+0000","last_undegraded":"2024-09-17T03:14:15.333916+0000","last_fullsized":"2024-09-17T03:14:15.333916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:13.543618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333254+0000","last_change":"2024-09-17T03:14:14.341872+0000","last_active":"2024-09-17T03:14:15.333254+0000","last_peered":"2024-09-17T03:14:15.333254+0000","last_clean":"2024-09-17T03:14:15.333254+0000","last_became_active":"2024-09-17T03:14:14.341555+0000","last_became_peered":"2024-09-17T03:14:14.341555+0000","last_unstale":"2024-09-17T03:14:15.333254+0000","last_undegraded":"2024-09-17T03:14:15.333254+0000","last_fullsized":"2024-09-17T03:14:15.333254+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:14.208665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332452+0000","last_change":"2024-09-17T03:14:13.325444+0000","last_active":"2024-09-17T03:14:15.332452+0000","last_peered":"2024-09-17T03:14:15.332452+0000","last_clean":"2024-09-17T03:14:15.332452+0000","last_became_active":"2024-09-17T03:14:13.325121+0000","last_became_peered":"2024-09-17T03:14:13.325121+0000","last_unstale":"2024-09-17T03:14:15.332452+0000","last_undegraded":"2024-09-17T03:14:15.332452+0000","last_fullsized":"2024-09-17T03:14:15.332452+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:55:19.372118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.390899+0000","last_change":"2024-09-17T03:14:14.341354+0000","last_active":"2024-09-17T03:14:16.390899+0000","last_peered":"2024-09-17T03:14:16.390899+0000","last_clean":"2024-09-17T03:14:16.390899+0000","last_became_active":"2024-09-17T03:14:14.341045+0000","last_became_peered":"2024-09-17T03:14:14.341045+0000","last_unstale":"2024-09-17T03:14:16.390899+0000","last_undegraded":"2024-09-17T03:14:16.390899+0000","last_fullsized":"2024-09-17T03:14:16.390899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:30.375807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333084+0000","last_change":"2024-09-17T03:14:13.321317+0000","last_active":"2024-09-17T03:14:15.333084+0000","last_peered":"2024-09-17T03:14:15.333084+0000","last_clean":"2024-09-17T03:14:15.333084+0000","last_became_active":"2024-09-17T03:14:13.320995+0000","last_became_peered":"2024-09-17T03:14:13.320995+0000","last_unstale":"2024-09-17T03:14:15.333084+0000","last_undegraded":"2024-09-17T03:14:15.333084+0000","last_fullsized":"2024-09-17T03:14:15.333084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:56.906875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333598+0000","last_change":"2024-09-17T03:14:14.342775+0000","last_active":"2024-09-17T03:14:15.333598+0000","last_peered":"2024-09-17T03:14:15.333598+0000","last_clean":"2024-09-17T03:14:15.333598+0000","last_became_active":"2024-09-17T03:14:14.342463+0000","last_became_peered":"2024-09-17T03:14:14.342463+0000","last_unstale":"2024-09-17T03:14:15.333598+0000","last_undegraded":"2024-09-17T03:14:15.333598+0000","last_fullsized":"2024-09-17T03:14:15.333598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:29:58.176940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333151+0000","last_change":"2024-09-17T03:14:13.324555+0000","last_active":"2024-09-17T03:14:15.333151+0000","last_peered":"2024-09-17T03:14:15.333151+0000","last_clean":"2024-09-17T03:14:15.333151+0000","last_became_active":"2024-09-17T03:14:13.323910+0000","last_became_peered":"2024-09-17T03:14:13.323910+0000","last_unstale":"2024-09-17T03:14:15.333151+0000","last_undegraded":"2024-09-17T03:14:15.333151+0000","last_fullsized":"2024-09-17T03:14:15.333151+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:48.923806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392232+0000","last_change":"2024-09-17T03:14:14.334852+0000","last_active":"2024-09-17T03:14:16.392232+0000","last_peered":"2024-09-17T03:14:16.392232+0000","last_clean":"2024-09-17T03:14:16.392232+0000","last_became_active":"2024-09-17T03:14:14.334469+0000","last_became_peered":"2024-09-17T03:14:14.334469+0000","last_unstale":"2024-09-17T03:14:16.392232+0000","last_undegraded":"2024-09-17T03:14:16.392232+0000","last_fullsized":"2024-09-17T03:14:16.392232+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:14.548392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333987+0000","last_change":"2024-09-17T03:14:13.322488+0000","last_active":"2024-09-17T03:14:15.333987+0000","last_peered":"2024-09-17T03:14:15.333987+0000","last_clean":"2024-09-17T03:14:15.333987+0000","last_became_active":"2024-09-17T03:14:13.322170+0000","last_became_peered":"2024-09-17T03:14:13.322170+0000","last_unstale":"2024-09-17T03:14:15.333987+0000","last_undegraded":"2024-09-17T03:14:15.333987+0000","last_fullsized":"2024-09-17T03:14:15.333987+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:13.679632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333094+0000","last_change":"2024-09-17T03:14:14.336281+0000","last_active":"2024-09-17T03:14:15.333094+0000","last_peered":"2024-09-17T03:14:15.333094+0000","last_clean":"2024-09-17T03:14:15.333094+0000","last_became_active":"2024-09-17T03:14:14.335986+0000","last_became_peered":"2024-09-17T03:14:14.335986+0000","last_unstale":"2024-09-17T03:14:15.333094+0000","last_undegraded":"2024-09-17T03:14:15.333094+0000","last_fullsized":"2024-09-17T03:14:15.333094+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:18.826069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334554+0000","last_change":"2024-09-17T03:14:13.318667+0000","last_active":"2024-09-17T03:14:15.334554+0000","last_peered":"2024-09-17T03:14:15.334554+0000","last_clean":"2024-09-17T03:14:15.334554+0000","last_became_active":"2024-09-17T03:14:13.318364+0000","last_became_peered":"2024-09-17T03:14:13.318364+0000","last_unstale":"2024-09-17T03:14:15.334554+0000","last_undegraded":"2024-09-17T03:14:15.334554+0000","last_fullsized":"2024-09-17T03:14:15.334554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:55.719231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333416+0000","last_change":"2024-09-17T03:14:14.334875+0000","last_active":"2024-09-17T03:14:15.333416+0000","last_peered":"2024-09-17T03:14:15.333416+0000","last_clean":"2024-09-17T03:14:15.333416+0000","last_became_active":"2024-09-17T03:14:14.334511+0000","last_became_peered":"2024-09-17T03:14:14.334511+0000","last_unstale":"2024-09-17T03:14:15.333416+0000","last_undegraded":"2024-09-17T03:14:15.333416+0000","last_fullsized":"2024-09-17T03:14:15.333416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:26.597012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392173+0000","last_change":"2024-09-17T03:14:13.330509+0000","last_active":"2024-09-17T03:14:16.392173+0000","last_peered":"2024-09-17T03:14:16.392173+0000","last_clean":"2024-09-17T03:14:16.392173+0000","last_became_active":"2024-09-17T03:14:13.330197+0000","last_became_peered":"2024-09-17T03:14:13.330197+0000","last_unstale":"2024-09-17T03:14:16.392173+0000","last_undegraded":"2024-09-17T03:14:16.392173+0000","last_fullsized":"2024-09-17T03:14:16.392173+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:53.797340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334097+0000","last_change":"2024-09-17T03:14:14.345943+0000","last_active":"2024-09-17T03:14:15.334097+0000","last_peered":"2024-09-17T03:14:15.334097+0000","last_clean":"2024-09-17T03:14:15.334097+0000","last_became_active":"2024-09-17T03:14:14.345637+0000","last_became_peered":"2024-09-17T03:14:14.345637+0000","last_unstale":"2024-09-17T03:14:15.334097+0000","last_undegraded":"2024-09-17T03:14:15.334097+0000","last_fullsized":"2024-09-17T03:14:15.334097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:23.070846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.390671+0000","last_change":"2024-09-17T03:14:04.862167+0000","last_active":"2024-09-17T03:14:16.390671+0000","last_peered":"2024-09-17T03:14:16.390671+0000","last_clean":"2024-09-17T03:14:16.390671+0000","last_became_active":"2024-09-17T03:14:04.861731+0000","last_became_peered":"2024-09-17T03:14:04.861731+0000","last_unstale":"2024-09-17T03:14:16.390671+0000","last_undegraded":"2024-09-17T03:14:16.390671+0000","last_fullsized":"2024-09-17T03:14:16.390671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_clean_scrub_stamp":"2024-09-17T03:14:03.847721+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:32.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333439+0000","last_change":"2024-09-17T03:14:13.318977+0000","last_active":"2024-09-17T03:14:15.333439+0000","last_peered":"2024-09-17T03:14:15.333439+0000","last_clean":"2024-09-17T03:14:15.333439+0000","last_became_active":"2024-09-17T03:14:13.318294+0000","last_became_peered":"2024-09-17T03:14:13.318294+0000","last_unstale":"2024-09-17T03:14:15.333439+0000","last_undegraded":"2024-09-17T03:14:15.333439+0000","last_fullsized":"2024-09-17T03:14:15.333439+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:39.634907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333413+0000","last_change":"2024-09-17T03:14:14.331745+0000","last_active":"2024-09-17T03:14:15.333413+0000","last_peered":"2024-09-17T03:14:15.333413+0000","last_clean":"2024-09-17T03:14:15.333413+0000","last_became_active":"2024-09-17T03:14:14.331207+0000","last_became_peered":"2024-09-17T03:14:14.331207+0000","last_unstale":"2024-09-17T03:14:15.333413+0000","last_undegraded":"2024-09-17T03:14:15.333413+0000","last_fullsized":"2024-09-17T03:14:15.333413+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:53:48.931242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332808+0000","last_change":"2024-09-17T03:14:13.324555+0000","last_active":"2024-09-17T03:14:15.332808+0000","last_peered":"2024-09-17T03:14:15.332808+0000","last_clean":"2024-09-17T03:14:15.332808+0000","last_became_active":"2024-09-17T03:14:13.324214+0000","last_became_peered":"2024-09-17T03:14:13.324214+0000","last_unstale":"2024-09-17T03:14:15.332808+0000","last_undegraded":"2024-09-17T03:14:15.332808+0000","last_fullsized":"2024-09-17T03:14:15.332808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:13.693714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391471+0000","last_change":"2024-09-17T03:14:14.339552+0000","last_active":"2024-09-17T03:14:16.391471+0000","last_peered":"2024-09-17T03:14:16.391471+0000","last_clean":"2024-09-17T03:14:16.391471+0000","last_became_active":"2024-09-17T03:14:14.339232+0000","last_became_peered":"2024-09-17T03:14:14.339232+0000","last_unstale":"2024-09-17T03:14:16.391471+0000","last_undegraded":"2024-09-17T03:14:16.391471+0000","last_fullsized":"2024-09-17T03:14:16.391471+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:31.035937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391696+0000","last_change":"2024-09-17T03:14:13.320582+0000","last_active":"2024-09-17T03:14:16.391696+0000","last_peered":"2024-09-17T03:14:16.391696+0000","last_clean":"2024-09-17T03:14:16.391696+0000","last_became_active":"2024-09-17T03:14:13.320217+0000","last_became_peered":"2024-09-17T03:14:13.320217+0000","last_unstale":"2024-09-17T03:14:16.391696+0000","last_undegraded":"2024-09-17T03:14:16.391696+0000","last_fullsized":"2024-09-17T03:14:16.391696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:34.405796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334392+0000","last_change":"2024-09-17T03:14:14.334667+0000","last_active":"2024-09-17T03:14:15.334392+0000","last_peered":"2024-09-17T03:14:15.334392+0000","last_clean":"2024-09-17T03:14:15.334392+0000","last_became_active":"2024-09-17T03:14:14.333112+0000","last_became_peered":"2024-09-17T03:14:14.333112+0000","last_unstale":"2024-09-17T03:14:15.334392+0000","last_undegraded":"2024-09-17T03:14:15.334392+0000","last_fullsized":"2024-09-17T03:14:15.334392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:17.642423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334025+0000","last_change":"2024-09-17T03:14:13.327887+0000","last_active":"2024-09-17T03:14:15.334025+0000","last_peered":"2024-09-17T03:14:15.334025+0000","last_clean":"2024-09-17T03:14:15.334025+0000","last_became_active":"2024-09-17T03:14:13.327514+0000","last_became_peered":"2024-09-17T03:14:13.327514+0000","last_unstale":"2024-09-17T03:14:15.334025+0000","last_undegraded":"2024-09-17T03:14:15.334025+0000","last_fullsized":"2024-09-17T03:14:15.334025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:10:41.615501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333005+0000","last_change":"2024-09-17T03:14:14.338404+0000","last_active":"2024-09-17T03:14:15.333005+0000","last_peered":"2024-09-17T03:14:15.333005+0000","last_clean":"2024-09-17T03:14:15.333005+0000","last_became_active":"2024-09-17T03:14:14.338120+0000","last_became_peered":"2024-09-17T03:14:14.338120+0000","last_unstale":"2024-09-17T03:14:15.333005+0000","last_undegraded":"2024-09-17T03:14:15.333005+0000","last_fullsized":"2024-09-17T03:14:15.333005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:17.576471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334665+0000","last_change":"2024-09-17T03:14:13.324534+0000","last_active":"2024-09-17T03:14:15.334665+0000","last_peered":"2024-09-17T03:14:15.334665+0000","last_clean":"2024-09-17T03:14:15.334665+0000","last_became_active":"2024-09-17T03:14:13.324173+0000","last_became_peered":"2024-09-17T03:14:13.324173+0000","last_unstale":"2024-09-17T03:14:15.334665+0000","last_undegraded":"2024-09-17T03:14:15.334665+0000","last_fullsized":"2024-09-17T03:14:15.334665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:20.601544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391770+0000","last_change":"2024-09-17T03:14:14.341535+0000","last_active":"2024-09-17T03:14:16.391770+0000","last_peered":"2024-09-17T03:14:16.391770+0000","last_clean":"2024-09-17T03:14:16.391770+0000","last_became_active":"2024-09-17T03:14:14.339526+0000","last_became_peered":"2024-09-17T03:14:14.339526+0000","last_unstale":"2024-09-17T03:14:16.391770+0000","last_undegraded":"2024-09-17T03:14:16.391770+0000","last_fullsized":"2024-09-17T03:14:16.391770+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:45.268231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334336+0000","last_change":"2024-09-17T03:14:13.321581+0000","last_active":"2024-09-17T03:14:15.334336+0000","last_peered":"2024-09-17T03:14:15.334336+0000","last_clean":"2024-09-17T03:14:15.334336+0000","last_became_active":"2024-09-17T03:14:13.321238+0000","last_became_peered":"2024-09-17T03:14:13.321238+0000","last_unstale":"2024-09-17T03:14:15.334336+0000","last_undegraded":"2024-09-17T03:14:15.334336+0000","last_fullsized":"2024-09-17T03:14:15.334336+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:57.200786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392107+0000","last_change":"2024-09-17T03:14:14.343574+0000","last_active":"2024-09-17T03:14:16.392107+0000","last_peered":"2024-09-17T03:14:16.392107+0000","last_clean":"2024-09-17T03:14:16.392107+0000","last_became_active":"2024-09-17T03:14:14.342586+0000","last_became_peered":"2024-09-17T03:14:14.342586+0000","last_unstale":"2024-09-17T03:14:16.392107+0000","last_undegraded":"2024-09-17T03:14:16.392107+0000","last_fullsized":"2024-09-17T03:14:16.392107+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:53:20.317357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334149+0000","last_change":"2024-09-17T03:14:13.321756+0000","last_active":"2024-09-17T03:14:15.334149+0000","last_peered":"2024-09-17T03:14:15.334149+0000","last_clean":"2024-09-17T03:14:15.334149+0000","last_became_active":"2024-09-17T03:14:13.321392+0000","last_became_peered":"2024-09-17T03:14:13.321392+0000","last_unstale":"2024-09-17T03:14:15.334149+0000","last_undegraded":"2024-09-17T03:14:15.334149+0000","last_fullsized":"2024-09-17T03:14:15.334149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:33.594511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333517+0000","last_change":"2024-09-17T03:14:14.341909+0000","last_active":"2024-09-17T03:14:15.333517+0000","last_peered":"2024-09-17T03:14:15.333517+0000","last_clean":"2024-09-17T03:14:15.333517+0000","last_became_active":"2024-09-17T03:14:14.341571+0000","last_became_peered":"2024-09-17T03:14:14.341571+0000","last_unstale":"2024-09-17T03:14:15.333517+0000","last_undegraded":"2024-09-17T03:14:15.333517+0000","last_fullsized":"2024-09-17T03:14:15.333517+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:41.776202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334199+0000","last_change":"2024-09-17T03:14:13.329358+0000","last_active":"2024-09-17T03:14:15.334199+0000","last_peered":"2024-09-17T03:14:15.334199+0000","last_clean":"2024-09-17T03:14:15.334199+0000","last_became_active":"2024-09-17T03:14:13.328836+0000","last_became_peered":"2024-09-17T03:14:13.328836+0000","last_unstale":"2024-09-17T03:14:15.334199+0000","last_undegraded":"2024-09-17T03:14:15.334199+0000","last_fullsized":"2024-09-17T03:14:15.334199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:44.263456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333819+0000","last_change":"2024-09-17T03:14:14.340502+0000","last_active":"2024-09-17T03:14:15.333819+0000","last_peered":"2024-09-17T03:14:15.333819+0000","last_clean":"2024-09-17T03:14:15.333819+0000","last_became_active":"2024-09-17T03:14:14.340215+0000","last_became_peered":"2024-09-17T03:14:14.340215+0000","last_unstale":"2024-09-17T03:14:15.333819+0000","last_undegraded":"2024-09-17T03:14:15.333819+0000","last_fullsized":"2024-09-17T03:14:15.333819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:45.283163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332057+0000","last_change":"2024-09-17T03:14:13.324045+0000","last_active":"2024-09-17T03:14:15.332057+0000","last_peered":"2024-09-17T03:14:15.332057+0000","last_clean":"2024-09-17T03:14:15.332057+0000","last_became_active":"2024-09-17T03:14:13.323025+0000","last_became_peered":"2024-09-17T03:14:13.323025+0000","last_unstale":"2024-09-17T03:14:15.332057+0000","last_undegraded":"2024-09-17T03:14:15.332057+0000","last_fullsized":"2024-09-17T03:14:15.332057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:02.157890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333680+0000","last_change":"2024-09-17T03:14:14.346858+0000","last_active":"2024-09-17T03:14:15.333680+0000","last_peered":"2024-09-17T03:14:15.333680+0000","last_clean":"2024-09-17T03:14:15.333680+0000","last_became_active":"2024-09-17T03:14:14.346576+0000","last_became_peered":"2024-09-17T03:14:14.346576+0000","last_unstale":"2024-09-17T03:14:15.333680+0000","last_undegraded":"2024-09-17T03:14:15.333680+0000","last_fullsized":"2024-09-17T03:14:15.333680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:38.080821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334132+0000","last_change":"2024-09-17T03:14:13.319558+0000","last_active":"2024-09-17T03:14:15.334132+0000","last_peered":"2024-09-17T03:14:15.334132+0000","last_clean":"2024-09-17T03:14:15.334132+0000","last_became_active":"2024-09-17T03:14:13.319236+0000","last_became_peered":"2024-09-17T03:14:13.319236+0000","last_unstale":"2024-09-17T03:14:15.334132+0000","last_undegraded":"2024-09-17T03:14:15.334132+0000","last_fullsized":"2024-09-17T03:14:15.334132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:25:39.260408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333163+0000","last_change":"2024-09-17T03:14:14.329694+0000","last_active":"2024-09-17T03:14:15.333163+0000","last_peered":"2024-09-17T03:14:15.333163+0000","last_clean":"2024-09-17T03:14:15.333163+0000","last_became_active":"2024-09-17T03:14:14.329289+0000","last_became_peered":"2024-09-17T03:14:14.329289+0000","last_unstale":"2024-09-17T03:14:15.333163+0000","last_undegraded":"2024-09-17T03:14:15.333163+0000","last_fullsized":"2024-09-17T03:14:15.333163+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:59.114330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392121+0000","last_change":"2024-09-17T03:14:13.323443+0000","last_active":"2024-09-17T03:14:16.392121+0000","last_peered":"2024-09-17T03:14:16.392121+0000","last_clean":"2024-09-17T03:14:16.392121+0000","last_became_active":"2024-09-17T03:14:13.323047+0000","last_became_peered":"2024-09-17T03:14:13.323047+0000","last_unstale":"2024-09-17T03:14:16.392121+0000","last_undegraded":"2024-09-17T03:14:16.392121+0000","last_fullsized":"2024-09-17T03:14:16.392121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:34.405796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392246+0000","last_change":"2024-09-17T03:14:14.336168+0000","last_active":"2024-09-17T03:14:16.392246+0000","last_peered":"2024-09-17T03:14:16.392246+0000","last_clean":"2024-09-17T03:14:16.392246+0000","last_became_active":"2024-09-17T03:14:14.335860+0000","last_became_peered":"2024-09-17T03:14:14.335860+0000","last_unstale":"2024-09-17T03:14:16.392246+0000","last_undegraded":"2024-09-17T03:14:16.392246+0000","last_fullsized":"2024-09-17T03:14:16.392246+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:40.435687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333326+0000","last_change":"2024-09-17T03:14:13.319349+0000","last_active":"2024-09-17T03:14:15.333326+0000","last_peered":"2024-09-17T03:14:15.333326+0000","last_clean":"2024-09-17T03:14:15.333326+0000","last_became_active":"2024-09-17T03:14:13.318677+0000","last_became_peered":"2024-09-17T03:14:13.318677+0000","last_unstale":"2024-09-17T03:14:15.333326+0000","last_undegraded":"2024-09-17T03:14:15.333326+0000","last_fullsized":"2024-09-17T03:14:15.333326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:55:19.372118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333793+0000","last_change":"2024-09-17T03:14:14.343651+0000","last_active":"2024-09-17T03:14:15.333793+0000","last_peered":"2024-09-17T03:14:15.333793+0000","last_clean":"2024-09-17T03:14:15.333793+0000","last_became_active":"2024-09-17T03:14:14.343219+0000","last_became_peered":"2024-09-17T03:14:14.343219+0000","last_unstale":"2024-09-17T03:14:15.333793+0000","last_undegraded":"2024-09-17T03:14:15.333793+0000","last_fullsized":"2024-09-17T03:14:15.333793+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:32.656876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333968+0000","last_change":"2024-09-17T03:14:14.347881+0000","last_active":"2024-09-17T03:14:15.333968+0000","last_peered":"2024-09-17T03:14:15.333968+0000","last_clean":"2024-09-17T03:14:15.333968+0000","last_became_active":"2024-09-17T03:14:14.346360+0000","last_became_peered":"2024-09-17T03:14:14.346360+0000","last_unstale":"2024-09-17T03:14:15.333968+0000","last_undegraded":"2024-09-17T03:14:15.333968+0000","last_fullsized":"2024-09-17T03:14:15.333968+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:21.993789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333484+0000","last_change":"2024-09-17T03:14:13.329489+0000","last_active":"2024-09-17T03:14:15.333484+0000","last_peered":"2024-09-17T03:14:15.333484+0000","last_clean":"2024-09-17T03:14:15.333484+0000","last_became_active":"2024-09-17T03:14:13.329141+0000","last_became_peered":"2024-09-17T03:14:13.329141+0000","last_unstale":"2024-09-17T03:14:15.333484+0000","last_undegraded":"2024-09-17T03:14:15.333484+0000","last_fullsized":"2024-09-17T03:14:15.333484+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:44.263456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391814+0000","last_change":"2024-09-17T03:14:14.337492+0000","last_active":"2024-09-17T03:14:16.391814+0000","last_peered":"2024-09-17T03:14:16.391814+0000","last_clean":"2024-09-17T03:14:16.391814+0000","last_became_active":"2024-09-17T03:14:14.337160+0000","last_became_peered":"2024-09-17T03:14:14.337160+0000","last_unstale":"2024-09-17T03:14:16.391814+0000","last_undegraded":"2024-09-17T03:14:16.391814+0000","last_fullsized":"2024-09-17T03:14:16.391814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:31.035937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333395+0000","last_change":"2024-09-17T03:14:13.323396+0000","last_active":"2024-09-17T03:14:15.333395+0000","last_peered":"2024-09-17T03:14:15.333395+0000","last_clean":"2024-09-17T03:14:15.333395+0000","last_became_active":"2024-09-17T03:14:13.322911+0000","last_became_peered":"2024-09-17T03:14:13.322911+0000","last_unstale":"2024-09-17T03:14:15.333395+0000","last_undegraded":"2024-09-17T03:14:15.333395+0000","last_fullsized":"2024-09-17T03:14:15.333395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:36.650878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.335094+0000","last_change":"2024-09-17T03:14:14.340359+0000","last_active":"2024-09-17T03:14:15.335094+0000","last_peered":"2024-09-17T03:14:15.335094+0000","last_clean":"2024-09-17T03:14:15.335094+0000","last_became_active":"2024-09-17T03:14:14.340060+0000","last_became_peered":"2024-09-17T03:14:14.340060+0000","last_unstale":"2024-09-17T03:14:15.335094+0000","last_undegraded":"2024-09-17T03:14:15.335094+0000","last_fullsized":"2024-09-17T03:14:15.335094+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:59:36.040787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334845+0000","last_change":"2024-09-17T03:14:13.317875+0000","last_active":"2024-09-17T03:14:15.334845+0000","last_peered":"2024-09-17T03:14:15.334845+0000","last_clean":"2024-09-17T03:14:15.334845+0000","last_became_active":"2024-09-17T03:14:13.317521+0000","last_became_peered":"2024-09-17T03:14:13.317521+0000","last_unstale":"2024-09-17T03:14:15.334845+0000","last_undegraded":"2024-09-17T03:14:15.334845+0000","last_fullsized":"2024-09-17T03:14:15.334845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:05.715850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.335048+0000","last_change":"2024-09-17T03:14:14.338400+0000","last_active":"2024-09-17T03:14:15.335048+0000","last_peered":"2024-09-17T03:14:15.335048+0000","last_clean":"2024-09-17T03:14:15.335048+0000","last_became_active":"2024-09-17T03:14:14.338003+0000","last_became_peered":"2024-09-17T03:14:14.338003+0000","last_unstale":"2024-09-17T03:14:15.335048+0000","last_undegraded":"2024-09-17T03:14:15.335048+0000","last_fullsized":"2024-09-17T03:14:15.335048+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:57:32.024812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333702+0000","last_change":"2024-09-17T03:14:13.324341+0000","last_active":"2024-09-17T03:14:15.333702+0000","last_peered":"2024-09-17T03:14:15.333702+0000","last_clean":"2024-09-17T03:14:15.333702+0000","last_became_active":"2024-09-17T03:14:13.323959+0000","last_became_peered":"2024-09-17T03:14:13.323959+0000","last_unstale":"2024-09-17T03:14:15.333702+0000","last_undegraded":"2024-09-17T03:14:15.333702+0000","last_fullsized":"2024-09-17T03:14:15.333702+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:18:06.456762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":82231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":82107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27844,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343996,"statfs":{"total":96636764160,"available":96608251904,"internally_reserved":0,"allocated":835584,"data_stored":672567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":82689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":245760,"data_stored":83719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":672545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509446,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":82163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":84391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:14:22.091 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T03:14:22.092 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-17T03:14:22.092 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-17T03:14:22.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.241+0000 7fd360d01640 1 -- 172.21.15.87:0/262925233 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd35c15ecf0 msgr2=0x7fd35c15f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:22.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.241+0000 7fd360d01640 1 --2- 172.21.15.87:0/262925233 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd35c15ecf0 0x7fd35c15f100 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd344009830 tx=0x7fd34402a510 comp rx=0 tx=0).stop 2024-09-17T03:14:22.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.241+0000 7fd360d01640 1 -- 172.21.15.87:0/262925233 shutdown_connections 2024-09-17T03:14:22.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.241+0000 7fd360d01640 1 --2- 172.21.15.87:0/262925233 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd35c15ecf0 0x7fd35c15f100 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.241+0000 7fd360d01640 1 -- 172.21.15.87:0/262925233 >> 172.21.15.87:0/262925233 conn(0x7fd35c0d4470 msgr2=0x7fd35c0d4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:22.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.241+0000 7fd360d01640 1 -- 172.21.15.87:0/262925233 shutdown_connections 2024-09-17T03:14:22.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.241+0000 7fd360d01640 1 -- 172.21.15.87:0/262925233 wait complete. 2024-09-17T03:14:22.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.243+0000 7fd360d01640 1 Processor -- start 2024-09-17T03:14:22.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.243+0000 7fd360d01640 1 -- start start 2024-09-17T03:14:22.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.245+0000 7fd360d01640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd35c0b5ef0 0x7fd35c1f5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:22.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.245+0000 7fd360d01640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd35c15c2d0 0x7fd35c0cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:22.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.245+0000 7fd360d01640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd35c15ecf0 0x7fd35c0cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:22.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.245+0000 7fd360d01640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c0d2fc0 con 0x7fd35c0b5ef0 2024-09-17T03:14:22.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.245+0000 7fd360d01640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd35c0d2e40 con 0x7fd35c15ecf0 2024-09-17T03:14:22.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.245+0000 7fd360d01640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd35c0d3140 con 0x7fd35c15c2d0 2024-09-17T03:14:22.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.245+0000 7fd35a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd35c0b5ef0 0x7fd35c1f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:22.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.245+0000 7fd35a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd35c0b5ef0 0x7fd35c1f5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40054/0 (socket says 172.21.15.87:40054) 2024-09-17T03:14:22.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.245+0000 7fd35a59c640 1 -- 172.21.15.87:0/3422490397 learned_addr learned my addr 172.21.15.87:0/3422490397 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:22.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.245+0000 7fd351d9b640 1 --2- 172.21.15.87:0/3422490397 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd35c15c2d0 0x7fd35c0cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:22.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.245+0000 7fd35ad9d640 1 --2- 172.21.15.87:0/3422490397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd35c15ecf0 0x7fd35c0cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:22.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.245+0000 7fd35a59c640 1 -- 172.21.15.87:0/3422490397 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd35c15c2d0 msgr2=0x7fd35c0cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:22.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.245+0000 7fd35a59c640 1 --2- 172.21.15.87:0/3422490397 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd35c15c2d0 0x7fd35c0cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.245+0000 7fd35a59c640 1 -- 172.21.15.87:0/3422490397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd35c15ecf0 msgr2=0x7fd35c0cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:22.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.245+0000 7fd35a59c640 1 --2- 172.21.15.87:0/3422490397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd35c15ecf0 0x7fd35c0cb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.245+0000 7fd35a59c640 1 -- 172.21.15.87:0/3422490397 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd35c1f58d0 con 0x7fd35c0b5ef0 2024-09-17T03:14:22.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.245+0000 7fd351d9b640 1 --2- 172.21.15.87:0/3422490397 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd35c15c2d0 0x7fd35c0cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:22.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.245+0000 7fd35ad9d640 1 --2- 172.21.15.87:0/3422490397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd35c15ecf0 0x7fd35c0cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:22.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.245+0000 7fd35a59c640 1 --2- 172.21.15.87:0/3422490397 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd35c0b5ef0 0x7fd35c1f5390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00b410 tx=0x7fd34c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:22.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.245+0000 7fd3537fe640 1 -- 172.21.15.87:0/3422490397 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c015020 con 0x7fd35c0b5ef0 2024-09-17T03:14:22.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.245+0000 7fd3537fe640 1 -- 172.21.15.87:0/3422490397 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd34c011920 con 0x7fd35c0b5ef0 2024-09-17T03:14:22.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.245+0000 7fd360d01640 1 -- 172.21.15.87:0/3422490397 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd35c0b9f00 con 0x7fd35c0b5ef0 2024-09-17T03:14:22.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.245+0000 7fd3537fe640 1 -- 172.21.15.87:0/3422490397 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c011d70 con 0x7fd35c0b5ef0 2024-09-17T03:14:22.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.245+0000 7fd360d01640 1 -- 172.21.15.87:0/3422490397 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd35c1484f0 con 0x7fd35c0b5ef0 2024-09-17T03:14:22.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.247+0000 7fd360d01640 1 -- 172.21.15.87:0/3422490397 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd35c0ce860 con 0x7fd35c0b5ef0 2024-09-17T03:14:22.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.247+0000 7fd3537fe640 1 -- 172.21.15.87:0/3422490397 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd34c018020 con 0x7fd35c0b5ef0 2024-09-17T03:14:22.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.247+0000 7fd3537fe640 1 --2- 172.21.15.87:0/3422490397 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd33c078bc0 0x7fd33c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:22.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.248+0000 7fd3537fe640 1 -- 172.21.15.87:0/3422490397 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd34c09ac00 con 0x7fd35c0b5ef0 2024-09-17T03:14:22.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.248+0000 7fd351d9b640 1 --2- 172.21.15.87:0/3422490397 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd33c078bc0 0x7fd33c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:22.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.248+0000 7fd351d9b640 1 --2- 172.21.15.87:0/3422490397 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd33c078bc0 0x7fd33c07b0d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd344034000 tx=0x7fd344004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:22.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.252+0000 7fd3537fe640 1 -- 172.21.15.87:0/3422490397 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd34c0a1080 con 0x7fd35c0b5ef0 2024-09-17T03:14:22.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.429+0000 7fd360d01640 1 -- 172.21.15.87:0/3422490397 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fd35c0d3640 con 0x7fd35c0b5ef0 2024-09-17T03:14:22.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.429+0000 7fd3537fe640 1 -- 172.21.15.87:0/3422490397 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v31) ==== 86+0+160 (secure 0 0 0) 0x7fd34c01f020 con 0x7fd35c0b5ef0 2024-09-17T03:14:22.428 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:22.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.431+0000 7fd360d01640 1 -- 172.21.15.87:0/3422490397 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd33c078bc0 msgr2=0x7fd33c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:22.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.431+0000 7fd360d01640 1 --2- 172.21.15.87:0/3422490397 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd33c078bc0 0x7fd33c07b0d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd344034000 tx=0x7fd344004730 comp rx=0 tx=0).stop 2024-09-17T03:14:22.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.431+0000 7fd360d01640 1 -- 172.21.15.87:0/3422490397 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd35c0b5ef0 msgr2=0x7fd35c1f5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:22.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.431+0000 7fd360d01640 1 --2- 172.21.15.87:0/3422490397 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd35c0b5ef0 0x7fd35c1f5390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd34c00b410 tx=0x7fd34c00b8d0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.431+0000 7fd360d01640 1 -- 172.21.15.87:0/3422490397 shutdown_connections 2024-09-17T03:14:22.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.431+0000 7fd360d01640 1 --2- 172.21.15.87:0/3422490397 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd33c078bc0 0x7fd33c07b0d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.431+0000 7fd360d01640 1 --2- 172.21.15.87:0/3422490397 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd35c15ecf0 0x7fd35c0cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.431+0000 7fd360d01640 1 --2- 172.21.15.87:0/3422490397 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd35c15c2d0 0x7fd35c0cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.431+0000 7fd360d01640 1 --2- 172.21.15.87:0/3422490397 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd35c0b5ef0 0x7fd35c1f5390 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.431+0000 7fd360d01640 1 -- 172.21.15.87:0/3422490397 >> 172.21.15.87:0/3422490397 conn(0x7fd35c0d4470 msgr2=0x7fd35c0d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:22.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.431+0000 7fd360d01640 1 -- 172.21.15.87:0/3422490397 shutdown_connections 2024-09-17T03:14:22.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.431+0000 7fd360d01640 1 -- 172.21.15.87:0/3422490397 wait complete. 2024-09-17T03:14:22.441 INFO:teuthology.orchestra.run.smithi087.stdout:[{"version":31,"timestamp":"2024-09-17T03:14:01.639252+0000","name":"","changes":[{"name":"osd.1/osd_mclock_max_capacity_iops_ssd","new_value":"4822.941320"}]}] 2024-09-17T03:14:22.441 INFO:tasks.ceph_manager:config epoch is 31 2024-09-17T03:14:22.442 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-17T03:14:22.442 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-17T03:14:22.442 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-17T03:14:22.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.554+0000 7fc8e3c30640 1 -- 172.21.15.87:0/942340188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc0cf430 msgr2=0x7fc8dc0b63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:22.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.554+0000 7fc8e3c30640 1 --2- 172.21.15.87:0/942340188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc0cf430 0x7fc8dc0b63e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0cfef0 tx=0x7fc8d402aa00 comp rx=0 tx=0).stop 2024-09-17T03:14:22.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.554+0000 7fc8e3c30640 1 -- 172.21.15.87:0/942340188 shutdown_connections 2024-09-17T03:14:22.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.554+0000 7fc8e3c30640 1 --2- 172.21.15.87:0/942340188 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc0cf430 0x7fc8dc0b63e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.554+0000 7fc8e3c30640 1 --2- 172.21.15.87:0/942340188 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc8dc0cf020 0x7fc8dc0b5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.554+0000 7fc8e3c30640 1 --2- 172.21.15.87:0/942340188 >> v2:172.21.15.149:3300/0 conn(0x7fc8dc15c5f0 0x7fc8dc15f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.554+0000 7fc8e3c30640 1 -- 172.21.15.87:0/942340188 >> 172.21.15.87:0/942340188 conn(0x7fc8dc069900 msgr2=0x7fc8dc0d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:22.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.554+0000 7fc8e3c30640 1 -- 172.21.15.87:0/942340188 shutdown_connections 2024-09-17T03:14:22.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.554+0000 7fc8e3c30640 1 -- 172.21.15.87:0/942340188 wait complete. 2024-09-17T03:14:22.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.556+0000 7fc8e3c30640 1 Processor -- start 2024-09-17T03:14:22.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.556+0000 7fc8e3c30640 1 -- start start 2024-09-17T03:14:22.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.557+0000 7fc8e3c30640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8dc0cf020 0x7fc8dc15aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:22.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.557+0000 7fc8e3c30640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc8dc0cf430 0x7fc8dc0cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:22.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.557+0000 7fc8e3c30640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc15c5f0 0x7fc8dc15dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:22.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.557+0000 7fc8e3c30640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc0d3190 con 0x7fc8dc15c5f0 2024-09-17T03:14:22.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.557+0000 7fc8e3c30640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc0d3010 con 0x7fc8dc0cf020 2024-09-17T03:14:22.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.557+0000 7fc8e3c30640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc8dc0d3310 con 0x7fc8dc0cf430 2024-09-17T03:14:22.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.557+0000 7fc8e21cd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc15c5f0 0x7fc8dc15dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:22.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.557+0000 7fc8e21cd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc15c5f0 0x7fc8dc15dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40072/0 (socket says 172.21.15.87:40072) 2024-09-17T03:14:22.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.557+0000 7fc8e21cd640 1 -- 172.21.15.87:0/37582130 learned_addr learned my addr 172.21.15.87:0/37582130 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:22.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.558+0000 7fc8e19cc640 1 --2- 172.21.15.87:0/37582130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8dc0cf020 0x7fc8dc15aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:22.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.558+0000 7fc8e11cb640 1 --2- 172.21.15.87:0/37582130 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc8dc0cf430 0x7fc8dc0cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:22.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.558+0000 7fc8e21cd640 1 -- 172.21.15.87:0/37582130 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc8dc0cf430 msgr2=0x7fc8dc0cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:22.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.558+0000 7fc8e21cd640 1 --2- 172.21.15.87:0/37582130 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc8dc0cf430 0x7fc8dc0cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.558+0000 7fc8e21cd640 1 -- 172.21.15.87:0/37582130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8dc0cf020 msgr2=0x7fc8dc15aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:22.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.558+0000 7fc8e21cd640 1 --2- 172.21.15.87:0/37582130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8dc0cf020 0x7fc8dc15aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.558+0000 7fc8e21cd640 1 -- 172.21.15.87:0/37582130 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8dc0ca030 con 0x7fc8dc15c5f0 2024-09-17T03:14:22.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.558+0000 7fc8e21cd640 1 --2- 172.21.15.87:0/37582130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc15c5f0 0x7fc8dc15dc20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc8d402aff0 tx=0x7fc8d4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:22.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.558+0000 7fc8d27fc640 1 -- 172.21.15.87:0/37582130 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4033050 con 0x7fc8dc15c5f0 2024-09-17T03:14:22.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.558+0000 7fc8d27fc640 1 -- 172.21.15.87:0/37582130 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8d4004450 con 0x7fc8dc15c5f0 2024-09-17T03:14:22.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.558+0000 7fc8e3c30640 1 -- 172.21.15.87:0/37582130 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8dc0ca310 con 0x7fc8dc15c5f0 2024-09-17T03:14:22.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.558+0000 7fc8d27fc640 1 -- 172.21.15.87:0/37582130 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4038450 con 0x7fc8dc15c5f0 2024-09-17T03:14:22.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.558+0000 7fc8e3c30640 1 -- 172.21.15.87:0/37582130 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8dc0ca8d0 con 0x7fc8dc15c5f0 2024-09-17T03:14:22.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.559+0000 7fc8e3c30640 1 -- 172.21.15.87:0/37582130 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8dc0c9da0 con 0x7fc8dc15c5f0 2024-09-17T03:14:22.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.560+0000 7fc8d27fc640 1 -- 172.21.15.87:0/37582130 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc8d4036070 con 0x7fc8dc15c5f0 2024-09-17T03:14:22.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.560+0000 7fc8d27fc640 1 --2- 172.21.15.87:0/37582130 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc8c0078b90 0x7fc8c007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:22.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.560+0000 7fc8d27fc640 1 -- 172.21.15.87:0/37582130 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc8d40bc050 con 0x7fc8dc15c5f0 2024-09-17T03:14:22.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.561+0000 7fc8e19cc640 1 --2- 172.21.15.87:0/37582130 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc8c0078b90 0x7fc8c007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:22.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.561+0000 7fc8e19cc640 1 --2- 172.21.15.87:0/37582130 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc8c0078b90 0x7fc8c007b0a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800bb50 tx=0x7fc8d800b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:22.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.565+0000 7fc8d27fc640 1 -- 172.21.15.87:0/37582130 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc8d4083e40 con 0x7fc8dc15c5f0 2024-09-17T03:14:22.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.785+0000 7fc8e3c30640 1 -- 172.21.15.87:0/37582130 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fc8dc15f890 con 0x7fc8dc15c5f0 2024-09-17T03:14:22.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.788+0000 7fc8d27fc640 1 -- 172.21.15.87:0/37582130 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+194719 (secure 0 0 0) 0x7fc8d4088870 con 0x7fc8dc15c5f0 2024-09-17T03:14:22.789 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:22.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.790+0000 7fc8e3c30640 1 -- 172.21.15.87:0/37582130 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc8c0078b90 msgr2=0x7fc8c007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:22.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.790+0000 7fc8e3c30640 1 --2- 172.21.15.87:0/37582130 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc8c0078b90 0x7fc8c007b0a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800bb50 tx=0x7fc8d800b2e0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.791+0000 7fc8e3c30640 1 -- 172.21.15.87:0/37582130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc15c5f0 msgr2=0x7fc8dc15dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:22.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.791+0000 7fc8e3c30640 1 --2- 172.21.15.87:0/37582130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc15c5f0 0x7fc8dc15dc20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc8d402aff0 tx=0x7fc8d4003f10 comp rx=0 tx=0).stop 2024-09-17T03:14:22.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.791+0000 7fc8e3c30640 1 -- 172.21.15.87:0/37582130 shutdown_connections 2024-09-17T03:14:22.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.791+0000 7fc8e3c30640 1 --2- 172.21.15.87:0/37582130 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc8c0078b90 0x7fc8c007b0a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.791+0000 7fc8e3c30640 1 --2- 172.21.15.87:0/37582130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc15c5f0 0x7fc8dc15dc20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.791+0000 7fc8e3c30640 1 --2- 172.21.15.87:0/37582130 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc8dc0cf430 0x7fc8dc0cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.791+0000 7fc8e3c30640 1 --2- 172.21.15.87:0/37582130 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8dc0cf020 0x7fc8dc15aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.791+0000 7fc8e3c30640 1 -- 172.21.15.87:0/37582130 >> 172.21.15.87:0/37582130 conn(0x7fc8dc069900 msgr2=0x7fc8dc161c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:22.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.791+0000 7fc8e3c30640 1 -- 172.21.15.87:0/37582130 shutdown_connections 2024-09-17T03:14:22.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.791+0000 7fc8e3c30640 1 -- 172.21.15.87:0/37582130 wait complete. 2024-09-17T03:14:22.800 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":5,"flags":0,"active_gid":4117,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":973360146},{"type":"v1","addr":"172.21.15.149:6801","nonce":973360146}]},"active_addr":"172.21.15.149:6801/973360146","active_change":"2024-09-17T03:13:53.408120+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4103,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.149:0","nonce":3857557206}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.149:0","nonce":2232603302}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.149:0","nonce":2394315361}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.149:0","nonce":3437112763}]}]} 2024-09-17T03:14:22.804 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-17T03:14:22.804 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-17T03:14:22.804 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:14:22.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.917+0000 7f8324fdf640 1 -- 172.21.15.87:0/2263395666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83200d0230 msgr2=0x7f83200d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:22.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.917+0000 7f8324fdf640 1 --2- 172.21.15.87:0/2263395666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83200d0230 0x7f83200d0640 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8318005fd0 tx=0x7f831802a870 comp rx=0 tx=0).stop 2024-09-17T03:14:22.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.917+0000 7f8324fdf640 1 -- 172.21.15.87:0/2263395666 shutdown_connections 2024-09-17T03:14:22.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.917+0000 7f8324fdf640 1 --2- 172.21.15.87:0/2263395666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83200d0230 0x7f83200d0640 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.917+0000 7f8324fdf640 1 --2- 172.21.15.87:0/2263395666 >> v2:172.21.15.149:3300/0 conn(0x7f83200b4470 0x7f832015c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.917+0000 7f8324fdf640 1 -- 172.21.15.87:0/2263395666 >> 172.21.15.87:0/2263395666 conn(0x7f8320069330 msgr2=0x7f83200ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:22.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.917+0000 7f8324fdf640 1 -- 172.21.15.87:0/2263395666 shutdown_connections 2024-09-17T03:14:22.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.917+0000 7f8324fdf640 1 -- 172.21.15.87:0/2263395666 wait complete. 2024-09-17T03:14:22.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.919+0000 7f8324fdf640 1 Processor -- start 2024-09-17T03:14:22.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.919+0000 7f8324fdf640 1 -- start start 2024-09-17T03:14:22.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.920+0000 7f8324fdf640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f83200b7f10 0x7f832013e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:22.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.920+0000 7f8324fdf640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83200d0230 0x7f83200b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:22.923 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.920+0000 7f8324fdf640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8320141250 0x7f8320140560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:22.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.920+0000 7f8324fdf640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f83200d0b80 con 0x7f83200d0230 2024-09-17T03:14:22.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.920+0000 7f8324fdf640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f83200b7d20 con 0x7f8320141250 2024-09-17T03:14:22.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.920+0000 7f8324fdf640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f83200d0d00 con 0x7f83200b7f10 2024-09-17T03:14:22.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.921+0000 7f8315d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83200d0230 0x7f83200b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:22.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.921+0000 7f8315d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83200d0230 0x7f83200b68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40082/0 (socket says 172.21.15.87:40082) 2024-09-17T03:14:22.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.921+0000 7f8315d9b640 1 -- 172.21.15.87:0/2520146278 learned_addr learned my addr 172.21.15.87:0/2520146278 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:22.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.921+0000 7f831e59c640 1 --2- 172.21.15.87:0/2520146278 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f83200b7f10 0x7f832013e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:22.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.921+0000 7f831ed9d640 1 --2- 172.21.15.87:0/2520146278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8320141250 0x7f8320140560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:22.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.921+0000 7f8315d9b640 1 -- 172.21.15.87:0/2520146278 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f83200b7f10 msgr2=0x7f832013e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:22.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.921+0000 7f8315d9b640 1 --2- 172.21.15.87:0/2520146278 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f83200b7f10 0x7f832013e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.921+0000 7f8315d9b640 1 -- 172.21.15.87:0/2520146278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8320141250 msgr2=0x7f8320140560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:22.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.921+0000 7f8315d9b640 1 --2- 172.21.15.87:0/2520146278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8320141250 0x7f8320140560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:22.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.921+0000 7f8315d9b640 1 -- 172.21.15.87:0/2520146278 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8320155640 con 0x7f83200d0230 2024-09-17T03:14:22.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.921+0000 7f831e59c640 1 --2- 172.21.15.87:0/2520146278 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f83200b7f10 0x7f832013e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:14:22.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.921+0000 7f8315d9b640 1 --2- 172.21.15.87:0/2520146278 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83200d0230 0x7f83200b68d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f830800d770 tx=0x7f830800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:22.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.921+0000 7f83177fe640 1 -- 172.21.15.87:0/2520146278 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8308017070 con 0x7f83200d0230 2024-09-17T03:14:22.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.921+0000 7f83177fe640 1 -- 172.21.15.87:0/2520146278 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8308009d20 con 0x7f83200d0230 2024-09-17T03:14:22.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.921+0000 7f8324fdf640 1 -- 172.21.15.87:0/2520146278 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8320155920 con 0x7f83200d0230 2024-09-17T03:14:22.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.921+0000 7f83177fe640 1 -- 172.21.15.87:0/2520146278 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83080106c0 con 0x7f83200d0230 2024-09-17T03:14:22.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.921+0000 7f8324fdf640 1 -- 172.21.15.87:0/2520146278 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f832013ea10 con 0x7f83200d0230 2024-09-17T03:14:22.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.922+0000 7f8324fdf640 1 -- 172.21.15.87:0/2520146278 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83200d11d0 con 0x7f83200d0230 2024-09-17T03:14:22.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.923+0000 7f83177fe640 1 -- 172.21.15.87:0/2520146278 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f830801a070 con 0x7f83200d0230 2024-09-17T03:14:22.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.923+0000 7f83177fe640 1 --2- 172.21.15.87:0/2520146278 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8300078bc0 0x7f830007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:22.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.924+0000 7f83177fe640 1 -- 172.21.15.87:0/2520146278 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f830809ae60 con 0x7f83200d0230 2024-09-17T03:14:22.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.924+0000 7f831e59c640 1 --2- 172.21.15.87:0/2520146278 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8300078bc0 0x7f830007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:22.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.924+0000 7f831e59c640 1 --2- 172.21.15.87:0/2520146278 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8300078bc0 0x7f830007b0d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8310007fd0 tx=0x7f8310009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:22.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:22.928+0000 7f83177fe640 1 -- 172.21.15.87:0/2520146278 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f83080632a0 con 0x7f83200d0230 2024-09-17T03:14:23.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.101+0000 7f8324fdf640 1 -- 172.21.15.87:0/2520146278 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f83200d1640 con 0x7f83200d0230 2024-09-17T03:14:23.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.102+0000 7f83177fe640 1 -- 172.21.15.87:0/2520146278 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16693 (secure 0 0 0) 0x7f8308062ba0 con 0x7f83200d0230 2024-09-17T03:14:23.101 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:23.102 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":23,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:14:15.323716+0000","last_up_change":"2024-09-17T03:14:01.833605+0000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:14:03.433404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:14:11.525547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T03:14:12.626396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6825","nonce":2430409749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6827","nonce":2430409749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6831","nonce":2430409749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6829","nonce":2430409749}]},"public_addr":"172.21.15.87:6825/2430409749","cluster_addr":"172.21.15.87:6827/2430409749","heartbeat_back_addr":"172.21.15.87:6831/2430409749","heartbeat_front_addr":"172.21.15.87:6829/2430409749","state":["exists","up"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6809","nonce":3467605435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6811","nonce":3467605435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6815","nonce":3467605435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6813","nonce":3467605435}]},"public_addr":"172.21.15.87:6809/3467605435","cluster_addr":"172.21.15.87:6811/3467605435","heartbeat_back_addr":"172.21.15.87:6815/3467605435","heartbeat_front_addr":"172.21.15.87:6813/3467605435","state":["exists","up"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6801","nonce":1450834766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6803","nonce":1450834766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6807","nonce":1450834766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6805","nonce":1450834766}]},"public_addr":"172.21.15.87:6801/1450834766","cluster_addr":"172.21.15.87:6803/1450834766","heartbeat_back_addr":"172.21.15.87:6807/1450834766","heartbeat_front_addr":"172.21.15.87:6805/1450834766","state":["exists","up"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6817","nonce":4253161216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6819","nonce":4253161216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6823","nonce":4253161216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6821","nonce":4253161216}]},"public_addr":"172.21.15.87:6817/4253161216","cluster_addr":"172.21.15.87:6819/4253161216","heartbeat_back_addr":"172.21.15.87:6823/4253161216","heartbeat_front_addr":"172.21.15.87:6821/4253161216","state":["exists","up"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6803","nonce":2414907089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6805","nonce":2414907089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6809","nonce":2414907089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6807","nonce":2414907089}]},"public_addr":"172.21.15.149:6803/2414907089","cluster_addr":"172.21.15.149:6805/2414907089","heartbeat_back_addr":"172.21.15.149:6809/2414907089","heartbeat_front_addr":"172.21.15.149:6807/2414907089","state":["exists","up"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6827","nonce":2182999530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6829","nonce":2182999530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6833","nonce":2182999530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6831","nonce":2182999530}]},"public_addr":"172.21.15.149:6827/2182999530","cluster_addr":"172.21.15.149:6829/2182999530","heartbeat_back_addr":"172.21.15.149:6833/2182999530","heartbeat_front_addr":"172.21.15.149:6831/2182999530","state":["exists","up"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6811","nonce":4289519146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6813","nonce":4289519146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6817","nonce":4289519146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6815","nonce":4289519146}]},"public_addr":"172.21.15.149:6811/4289519146","cluster_addr":"172.21.15.149:6813/4289519146","heartbeat_back_addr":"172.21.15.149:6817/4289519146","heartbeat_front_addr":"172.21.15.149:6815/4289519146","state":["exists","up"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6819","nonce":2771010477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6821","nonce":2771010477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6825","nonce":2771010477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6823","nonce":2771010477}]},"public_addr":"172.21.15.149:6819/2771010477","cluster_addr":"172.21.15.149:6821/2771010477","heartbeat_back_addr":"172.21.15.149:6825/2771010477","heartbeat_front_addr":"172.21.15.149:6823/2771010477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.313297+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.642861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.654781+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.302606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.385791+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.915266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.121365+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.394432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:14:23.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.104+0000 7f8324fdf640 1 -- 172.21.15.87:0/2520146278 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8300078bc0 msgr2=0x7f830007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.104+0000 7f8324fdf640 1 --2- 172.21.15.87:0/2520146278 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8300078bc0 0x7f830007b0d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8310007fd0 tx=0x7f8310009040 comp rx=0 tx=0).stop 2024-09-17T03:14:23.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.104+0000 7f8324fdf640 1 -- 172.21.15.87:0/2520146278 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83200d0230 msgr2=0x7f83200b68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.104+0000 7f8324fdf640 1 --2- 172.21.15.87:0/2520146278 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83200d0230 0x7f83200b68d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f830800d770 tx=0x7f830800dc30 comp rx=0 tx=0).stop 2024-09-17T03:14:23.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.105+0000 7f8324fdf640 1 -- 172.21.15.87:0/2520146278 shutdown_connections 2024-09-17T03:14:23.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.105+0000 7f8324fdf640 1 --2- 172.21.15.87:0/2520146278 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8300078bc0 0x7f830007b0d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.105+0000 7f8324fdf640 1 --2- 172.21.15.87:0/2520146278 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8320141250 0x7f8320140560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.105+0000 7f8324fdf640 1 --2- 172.21.15.87:0/2520146278 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83200d0230 0x7f83200b68d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.105+0000 7f8324fdf640 1 --2- 172.21.15.87:0/2520146278 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f83200b7f10 0x7f832013e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.105+0000 7f8324fdf640 1 -- 172.21.15.87:0/2520146278 >> 172.21.15.87:0/2520146278 conn(0x7f8320069330 msgr2=0x7f83200d2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:23.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.105+0000 7f8324fdf640 1 -- 172.21.15.87:0/2520146278 shutdown_connections 2024-09-17T03:14:23.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.105+0000 7f8324fdf640 1 -- 172.21.15.87:0/2520146278 wait complete. 2024-09-17T03:14:23.115 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-17T03:14:23.115 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:14:23.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.229+0000 7fc6ae19d640 1 -- 172.21.15.87:0/1096871477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc694003a30 msgr2=0x7fc694003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.229+0000 7fc6ae19d640 1 --2- 172.21.15.87:0/1096871477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc694003a30 0x7fc694003ea0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4001660 tx=0x7fc694005830 comp rx=0 tx=0).stop 2024-09-17T03:14:23.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.229+0000 7fc6ad99c640 1 -- 172.21.15.87:0/1096871477 reap_dead start 2024-09-17T03:14:23.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.229+0000 7fc6ae19d640 1 -- 172.21.15.87:0/1096871477 shutdown_connections 2024-09-17T03:14:23.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.229+0000 7fc6ae19d640 1 -- 172.21.15.87:0/1096871477 >> 172.21.15.87:0/1096871477 conn(0x7fc6a8069860 msgr2=0x7fc6a809d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:23.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.229+0000 7fc6ae19d640 1 -- 172.21.15.87:0/1096871477 shutdown_connections 2024-09-17T03:14:23.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.229+0000 7fc6ae19d640 1 -- 172.21.15.87:0/1096871477 wait complete. 2024-09-17T03:14:23.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.231+0000 7fc6ae19d640 1 Processor -- start 2024-09-17T03:14:23.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.231+0000 7fc6ae19d640 1 -- start start 2024-09-17T03:14:23.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.232+0000 7fc6ae19d640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc694003a30 0x7fc6a81f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.232+0000 7fc6ae19d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6a81f7a30 0x7fc6a80d0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.232+0000 7fc6ae19d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6a80d1230 0x7fc6a806daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.232+0000 7fc6ae19d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a80b7e50 con 0x7fc6a81f7a30 2024-09-17T03:14:23.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.232+0000 7fc6ae19d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc6a80b7cd0 con 0x7fc6a80d1230 2024-09-17T03:14:23.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.232+0000 7fc6ae19d640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc6a80b7fd0 con 0x7fc694003a30 2024-09-17T03:14:23.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.232+0000 7fc6ac99a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6a81f7a30 0x7fc6a80d0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.232+0000 7fc6ac99a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6a81f7a30 0x7fc6a80d0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40094/0 (socket says 172.21.15.87:40094) 2024-09-17T03:14:23.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.232+0000 7fc6ac99a640 1 -- 172.21.15.87:0/491932158 learned_addr learned my addr 172.21.15.87:0/491932158 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:23.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.233+0000 7fc6ad19b640 1 --2- 172.21.15.87:0/491932158 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc694003a30 0x7fc6a81f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.233+0000 7fc6ad99c640 1 --2- 172.21.15.87:0/491932158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6a80d1230 0x7fc6a806daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.233+0000 7fc6ac99a640 1 -- 172.21.15.87:0/491932158 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc694003a30 msgr2=0x7fc6a81f5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.233+0000 7fc6ac99a640 1 --2- 172.21.15.87:0/491932158 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc694003a30 0x7fc6a81f5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.233+0000 7fc6ac99a640 1 -- 172.21.15.87:0/491932158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6a80d1230 msgr2=0x7fc6a806daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.233+0000 7fc6ac99a640 1 --2- 172.21.15.87:0/491932158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6a80d1230 0x7fc6a806daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.233+0000 7fc6ac99a640 1 -- 172.21.15.87:0/491932158 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6a80d1640 con 0x7fc6a81f7a30 2024-09-17T03:14:23.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.233+0000 7fc6ac99a640 1 --2- 172.21.15.87:0/491932158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6a81f7a30 0x7fc6a80d0cf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc69800e5e0 tx=0x7fc69800eaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.233+0000 7fc69dffb640 1 -- 172.21.15.87:0/491932158 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc698015070 con 0x7fc6a81f7a30 2024-09-17T03:14:23.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.233+0000 7fc69dffb640 1 -- 172.21.15.87:0/491932158 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc69800ca80 con 0x7fc6a81f7a30 2024-09-17T03:14:23.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.233+0000 7fc6ae19d640 1 -- 172.21.15.87:0/491932158 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6a81f8610 con 0x7fc6a81f7a30 2024-09-17T03:14:23.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.233+0000 7fc69dffb640 1 -- 172.21.15.87:0/491932158 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc698011460 con 0x7fc6a81f7a30 2024-09-17T03:14:23.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.233+0000 7fc6ae19d640 1 -- 172.21.15.87:0/491932158 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6a81f8bd0 con 0x7fc6a81f7a30 2024-09-17T03:14:23.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.234+0000 7fc6ae19d640 1 -- 172.21.15.87:0/491932158 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6a80b8560 con 0x7fc6a81f7a30 2024-09-17T03:14:23.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.235+0000 7fc69dffb640 1 -- 172.21.15.87:0/491932158 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc698018070 con 0x7fc6a81f7a30 2024-09-17T03:14:23.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.235+0000 7fc69dffb640 1 --2- 172.21.15.87:0/491932158 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc68c078b90 0x7fc68c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.235+0000 7fc69dffb640 1 -- 172.21.15.87:0/491932158 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc69809b130 con 0x7fc6a81f7a30 2024-09-17T03:14:23.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.236+0000 7fc6ad19b640 1 --2- 172.21.15.87:0/491932158 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc68c078b90 0x7fc68c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.236+0000 7fc6ad19b640 1 --2- 172.21.15.87:0/491932158 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc68c078b90 0x7fc68c07b0a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4032000 tx=0x7fc6a4001190 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.238 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.240+0000 7fc69dffb640 1 -- 172.21.15.87:0/491932158 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc6a80d1640 con 0x7fc6a81f7a30 2024-09-17T03:14:23.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.414+0000 7fc6ae19d640 1 -- 172.21.15.87:0/491932158 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc6a81f83a0 con 0x7fc6a81f7a30 2024-09-17T03:14:23.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.415+0000 7fc69dffb640 1 -- 172.21.15.87:0/491932158 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16693 (secure 0 0 0) 0x7fc698063570 con 0x7fc6a81f7a30 2024-09-17T03:14:23.414 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:23.414 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":23,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:14:15.323716+0000","last_up_change":"2024-09-17T03:14:01.833605+0000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:14:03.433404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:14:11.525547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T03:14:12.626396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6825","nonce":2430409749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6827","nonce":2430409749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6831","nonce":2430409749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6829","nonce":2430409749}]},"public_addr":"172.21.15.87:6825/2430409749","cluster_addr":"172.21.15.87:6827/2430409749","heartbeat_back_addr":"172.21.15.87:6831/2430409749","heartbeat_front_addr":"172.21.15.87:6829/2430409749","state":["exists","up"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6809","nonce":3467605435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6811","nonce":3467605435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6815","nonce":3467605435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6813","nonce":3467605435}]},"public_addr":"172.21.15.87:6809/3467605435","cluster_addr":"172.21.15.87:6811/3467605435","heartbeat_back_addr":"172.21.15.87:6815/3467605435","heartbeat_front_addr":"172.21.15.87:6813/3467605435","state":["exists","up"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6801","nonce":1450834766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6803","nonce":1450834766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6807","nonce":1450834766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6805","nonce":1450834766}]},"public_addr":"172.21.15.87:6801/1450834766","cluster_addr":"172.21.15.87:6803/1450834766","heartbeat_back_addr":"172.21.15.87:6807/1450834766","heartbeat_front_addr":"172.21.15.87:6805/1450834766","state":["exists","up"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6817","nonce":4253161216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6819","nonce":4253161216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6823","nonce":4253161216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6821","nonce":4253161216}]},"public_addr":"172.21.15.87:6817/4253161216","cluster_addr":"172.21.15.87:6819/4253161216","heartbeat_back_addr":"172.21.15.87:6823/4253161216","heartbeat_front_addr":"172.21.15.87:6821/4253161216","state":["exists","up"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6803","nonce":2414907089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6805","nonce":2414907089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6809","nonce":2414907089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6807","nonce":2414907089}]},"public_addr":"172.21.15.149:6803/2414907089","cluster_addr":"172.21.15.149:6805/2414907089","heartbeat_back_addr":"172.21.15.149:6809/2414907089","heartbeat_front_addr":"172.21.15.149:6807/2414907089","state":["exists","up"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6827","nonce":2182999530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6829","nonce":2182999530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6833","nonce":2182999530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6831","nonce":2182999530}]},"public_addr":"172.21.15.149:6827/2182999530","cluster_addr":"172.21.15.149:6829/2182999530","heartbeat_back_addr":"172.21.15.149:6833/2182999530","heartbeat_front_addr":"172.21.15.149:6831/2182999530","state":["exists","up"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6811","nonce":4289519146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6813","nonce":4289519146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6817","nonce":4289519146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6815","nonce":4289519146}]},"public_addr":"172.21.15.149:6811/4289519146","cluster_addr":"172.21.15.149:6813/4289519146","heartbeat_back_addr":"172.21.15.149:6817/4289519146","heartbeat_front_addr":"172.21.15.149:6815/4289519146","state":["exists","up"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6819","nonce":2771010477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6821","nonce":2771010477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6825","nonce":2771010477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6823","nonce":2771010477}]},"public_addr":"172.21.15.149:6819/2771010477","cluster_addr":"172.21.15.149:6821/2771010477","heartbeat_back_addr":"172.21.15.149:6825/2771010477","heartbeat_front_addr":"172.21.15.149:6823/2771010477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.313297+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.642861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.654781+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.302606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.385791+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.915266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.121365+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.394432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:14:23.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.417+0000 7fc6ae19d640 1 -- 172.21.15.87:0/491932158 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc68c078b90 msgr2=0x7fc68c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.417+0000 7fc6ae19d640 1 --2- 172.21.15.87:0/491932158 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc68c078b90 0x7fc68c07b0a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4032000 tx=0x7fc6a4001190 comp rx=0 tx=0).stop 2024-09-17T03:14:23.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.417+0000 7fc6ae19d640 1 -- 172.21.15.87:0/491932158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6a81f7a30 msgr2=0x7fc6a80d0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.417+0000 7fc6ae19d640 1 --2- 172.21.15.87:0/491932158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6a81f7a30 0x7fc6a80d0cf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc69800e5e0 tx=0x7fc69800eaa0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.417+0000 7fc6ae19d640 1 -- 172.21.15.87:0/491932158 shutdown_connections 2024-09-17T03:14:23.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.417+0000 7fc6ae19d640 1 --2- 172.21.15.87:0/491932158 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc68c078b90 0x7fc68c07b0a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.417+0000 7fc6ae19d640 1 --2- 172.21.15.87:0/491932158 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc6a80d1230 0x7fc6a806daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.417+0000 7fc6ae19d640 1 --2- 172.21.15.87:0/491932158 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6a81f7a30 0x7fc6a80d0cf0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.417+0000 7fc6ae19d640 1 --2- 172.21.15.87:0/491932158 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc694003a30 0x7fc6a81f5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.417+0000 7fc6ae19d640 1 -- 172.21.15.87:0/491932158 >> 172.21.15.87:0/491932158 conn(0x7fc6a8069860 msgr2=0x7fc6a806ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:23.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.417+0000 7fc6ae19d640 1 -- 172.21.15.87:0/491932158 shutdown_connections 2024-09-17T03:14:23.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.417+0000 7fc6ae19d640 1 -- 172.21.15.87:0/491932158 wait complete. 2024-09-17T03:14:23.427 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-17T03:14:23.518 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-17T03:14:23.518 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-17T03:14:23.519 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-17T03:14:23.520 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-17T03:14:23.520 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-17T03:14:23.521 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-17T03:14:23.522 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-17T03:14:23.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.738+0000 7f3285d9b640 1 -- 172.21.15.87:0/3828929590 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3280030b10 con 0x7f3288073b70 2024-09-17T03:14:23.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.739+0000 7f280990c640 1 -- 172.21.15.87:0/787206717 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f27fc055280 msgr2=0x7f27fc009010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.739+0000 7f280990c640 1 --2- 172.21.15.87:0/787206717 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f27fc055280 0x7f27fc009010 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f27f8005fd0 tx=0x7f27f802a660 comp rx=0 tx=0).stop 2024-09-17T03:14:23.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.739+0000 7f280990c640 1 -- 172.21.15.87:0/787206717 shutdown_connections 2024-09-17T03:14:23.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.739+0000 7f280990c640 1 --2- 172.21.15.87:0/787206717 >> v2:172.21.15.149:3300/0 conn(0x7f27fc036770 0x7f27fc0557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.739+0000 7f280990c640 1 --2- 172.21.15.87:0/787206717 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f27fc055280 0x7f27fc009010 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.739+0000 7f280990c640 1 -- 172.21.15.87:0/787206717 >> 172.21.15.87:0/787206717 conn(0x7f27fc01eae0 msgr2=0x7f27fc03a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:23.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.739+0000 7f280990c640 1 -- 172.21.15.87:0/787206717 shutdown_connections 2024-09-17T03:14:23.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.740+0000 7f328d0b6640 1 -- 172.21.15.87:0/3828929590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3288073b70 msgr2=0x7f32880aec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.740+0000 7f328d0b6640 1 --2- 172.21.15.87:0/3828929590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3288073b70 0x7f32880aec30 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3280005fd0 tx=0x7f328002a940 comp rx=0 tx=0).stop 2024-09-17T03:14:23.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.740+0000 7f328d0b6640 1 -- 172.21.15.87:0/3828929590 shutdown_connections 2024-09-17T03:14:23.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.740+0000 7f328d0b6640 1 --2- 172.21.15.87:0/3828929590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3288073b70 0x7f32880aec30 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.740+0000 7f328d0b6640 1 -- 172.21.15.87:0/3828929590 >> 172.21.15.87:0/3828929590 conn(0x7f32880d08c0 msgr2=0x7f32880d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:23.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.740+0000 7f328d0b6640 1 -- 172.21.15.87:0/3828929590 shutdown_connections 2024-09-17T03:14:23.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.740+0000 7fb6c9dda640 1 -- 172.21.15.87:0/1872098829 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb6bc055ee0 msgr2=0x7fb6bc0dfe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.740+0000 7fb6c9dda640 1 --2- 172.21.15.87:0/1872098829 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb6bc055ee0 0x7fb6bc0dfe80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8009830 tx=0x7fb6b802a480 comp rx=0 tx=0).stop 2024-09-17T03:14:23.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.740+0000 7fb6c9dda640 1 -- 172.21.15.87:0/1872098829 shutdown_connections 2024-09-17T03:14:23.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.740+0000 7fb6c9dda640 1 --2- 172.21.15.87:0/1872098829 >> v2:172.21.15.87:3300/0 conn(0x7fb6bc008fa0 0x7fb6bc053410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.740+0000 7fb6c9dda640 1 --2- 172.21.15.87:0/1872098829 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb6bc055ee0 0x7fb6bc0dfe80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.740+0000 7f328d0b6640 1 -- 172.21.15.87:0/3828929590 wait complete. 2024-09-17T03:14:23.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.740+0000 7fb6c9dda640 1 -- 172.21.15.87:0/1872098829 >> 172.21.15.87:0/1872098829 conn(0x7fb6bc01eae0 msgr2=0x7fb6bc03a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:23.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.740+0000 7fb6c9dda640 1 -- 172.21.15.87:0/1872098829 shutdown_connections 2024-09-17T03:14:23.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.740+0000 7fb6c9dda640 1 -- 172.21.15.87:0/1872098829 wait complete. 2024-09-17T03:14:23.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.740+0000 7fd02e6cc640 1 -- 172.21.15.87:0/401787620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd020035220 msgr2=0x7fd020035630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.740+0000 7fd02e6cc640 1 --2- 172.21.15.87:0/401787620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd020035220 0x7fd020035630 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd020038420 tx=0x7fd01402a410 comp rx=0 tx=0).stop 2024-09-17T03:14:23.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.740+0000 7fd02e6cc640 1 -- 172.21.15.87:0/401787620 shutdown_connections 2024-09-17T03:14:23.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.740+0000 7fd02e6cc640 1 --2- 172.21.15.87:0/401787620 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd020035220 0x7fd020035630 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.740+0000 7fd02e6cc640 1 --2- 172.21.15.87:0/401787620 >> v2:172.21.15.87:3300/0 conn(0x7fd0200dfe70 0x7fd020008d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.740+0000 7fd02e6cc640 1 -- 172.21.15.87:0/401787620 >> 172.21.15.87:0/401787620 conn(0x7fd02001eb10 msgr2=0x7fd02003a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:23.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.740+0000 7fd02e6cc640 1 -- 172.21.15.87:0/401787620 shutdown_connections 2024-09-17T03:14:23.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.740+0000 7fd02e6cc640 1 -- 172.21.15.87:0/401787620 wait complete. 2024-09-17T03:14:23.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.741+0000 7f114c23f640 1 -- 172.21.15.87:0/7425457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f113c009ba0 msgr2=0x7f113c0df180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.741+0000 7f114c23f640 1 --2- 172.21.15.87:0/7425457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f113c009ba0 0x7f113c0df180 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1140009820 tx=0x7f114002a500 comp rx=0 tx=0).stop 2024-09-17T03:14:23.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.741+0000 7f114c23f640 1 -- 172.21.15.87:0/7425457 shutdown_connections 2024-09-17T03:14:23.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.741+0000 7f114c23f640 1 --2- 172.21.15.87:0/7425457 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f113c005c60 0x7f113c036430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.741+0000 7f114c23f640 1 --2- 172.21.15.87:0/7425457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f113c009ba0 0x7f113c0df180 secure :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1140009820 tx=0x7f114002a500 comp rx=0 tx=0).stop 2024-09-17T03:14:23.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.741+0000 7f114c23f640 1 -- 172.21.15.87:0/7425457 >> 172.21.15.87:0/7425457 conn(0x7f113c051420 msgr2=0x7f113c051860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:23.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.740+0000 7f280990c640 1 -- 172.21.15.87:0/787206717 wait complete. 2024-09-17T03:14:23.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.742+0000 7f304ae69640 1 -- 172.21.15.87:0/3734299470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f304415aed0 msgr2=0x7f30440cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.742+0000 7f304ae69640 1 --2- 172.21.15.87:0/3734299470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f304415aed0 0x7f30440cc7c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f303c005fd0 tx=0x7f303c02a540 comp rx=0 tx=0).stop 2024-09-17T03:14:23.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.742+0000 7f30437fe640 1 -- 172.21.15.87:0/3734299470 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c0047a0 con 0x7f304415aed0 2024-09-17T03:14:23.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.742+0000 7fe01e7fc640 1 -- 172.21.15.87:0/994800713 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe010031bf0 con 0x7fe0200b9770 2024-09-17T03:14:23.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.742+0000 7f304ae69640 1 -- 172.21.15.87:0/3734299470 shutdown_connections 2024-09-17T03:14:23.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.742+0000 7f304ae69640 1 --2- 172.21.15.87:0/3734299470 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f304415b650 0x7f304415ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.742+0000 7f304ae69640 1 --2- 172.21.15.87:0/3734299470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f304415aed0 0x7f30440cc7c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.742+0000 7f304ae69640 1 -- 172.21.15.87:0/3734299470 >> 172.21.15.87:0/3734299470 conn(0x7f304406ee70 msgr2=0x7f304406f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:23.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.742+0000 7f280990c640 1 Processor -- start 2024-09-17T03:14:23.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.742+0000 7f280990c640 1 -- start start 2024-09-17T03:14:23.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.742+0000 7f114c23f640 1 -- 172.21.15.87:0/7425457 shutdown_connections 2024-09-17T03:14:23.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.742+0000 7fe025e8b640 1 -- 172.21.15.87:0/994800713 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0200b9770 msgr2=0x7fe0200d4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.742+0000 7fe025e8b640 1 --2- 172.21.15.87:0/994800713 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0200b9770 0x7fe0200d4f60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe02008d0f0 tx=0x7fe01002a3e0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.742+0000 7fe025e8b640 1 -- 172.21.15.87:0/994800713 shutdown_connections 2024-09-17T03:14:23.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.742+0000 7fe025e8b640 1 --2- 172.21.15.87:0/994800713 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe0200b3c30 0x7fe0200b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.742+0000 7fe025e8b640 1 --2- 172.21.15.87:0/994800713 >> v2:172.21.15.149:3300/0 conn(0x7fe0200cfca0 0x7fe0200d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.742+0000 7fe025e8b640 1 --2- 172.21.15.87:0/994800713 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0200b9770 0x7fe0200d4f60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.742+0000 7fe025e8b640 1 -- 172.21.15.87:0/994800713 >> 172.21.15.87:0/994800713 conn(0x7fe0200d4440 msgr2=0x7fe0200d4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:23.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.742+0000 7f114c23f640 1 -- 172.21.15.87:0/7425457 wait complete. 2024-09-17T03:14:23.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.743+0000 7fd02e6cc640 1 Processor -- start 2024-09-17T03:14:23.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.743+0000 7fb6c9dda640 1 Processor -- start 2024-09-17T03:14:23.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.743+0000 7f4f8292b640 1 -- 172.21.15.87:0/4261099069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f6c0048b0 msgr2=0x7f4f6c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.743+0000 7f4f8292b640 1 --2- 172.21.15.87:0/4261099069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f6c0048b0 0x7f4f6c004cc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800a610 tx=0x7f4f6c006940 comp rx=0 tx=0).stop 2024-09-17T03:14:23.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.743+0000 7f328d0b6640 1 Processor -- start 2024-09-17T03:14:23.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.743+0000 7f328d0b6640 1 -- start start 2024-09-17T03:14:23.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.743+0000 7f304ae69640 1 -- 172.21.15.87:0/3734299470 shutdown_connections 2024-09-17T03:14:23.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.743+0000 7f304ae69640 1 -- 172.21.15.87:0/3734299470 wait complete. 2024-09-17T03:14:23.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.743+0000 7f4f8292b640 1 -- 172.21.15.87:0/4261099069 shutdown_connections 2024-09-17T03:14:23.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.744+0000 7f4f8292b640 1 --2- 172.21.15.87:0/4261099069 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f6c0048b0 0x7f4f6c004cc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.744+0000 7fe025e8b640 1 -- 172.21.15.87:0/994800713 shutdown_connections 2024-09-17T03:14:23.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.744+0000 7fd02e6cc640 1 -- start start 2024-09-17T03:14:23.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.744+0000 7f4f8292b640 1 -- 172.21.15.87:0/4261099069 >> 172.21.15.87:0/4261099069 conn(0x7f4f74051420 msgr2=0x7f4f74051860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:23.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.744+0000 7f4f8292b640 1 -- 172.21.15.87:0/4261099069 shutdown_connections 2024-09-17T03:14:23.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.744+0000 7f4f8292b640 1 -- 172.21.15.87:0/4261099069 wait complete. 2024-09-17T03:14:23.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.744+0000 7f280990c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27fc036770 0x7f27fc0e6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.744+0000 7f280990c640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f27fc036cc0 0x7f27fc181e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.744+0000 7f280990c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27fc055280 0x7f27fc1823c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.744+0000 7f280990c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc055d10 con 0x7f27fc055280 2024-09-17T03:14:23.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.744+0000 7f280990c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f27fc055e90 con 0x7f27fc036770 2024-09-17T03:14:23.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.744+0000 7f280990c640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f27fc056010 con 0x7f27fc036cc0 2024-09-17T03:14:23.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.744+0000 7fb6c9dda640 1 -- start start 2024-09-17T03:14:23.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.745+0000 7f114c23f640 1 Processor -- start 2024-09-17T03:14:23.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.744+0000 7fe025e8b640 1 -- 172.21.15.87:0/994800713 wait complete. 2024-09-17T03:14:23.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.745+0000 7f280890a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27fc036770 0x7f27fc0e6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.745+0000 7f2803fff640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f27fc036cc0 0x7f27fc181e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.745+0000 7f114c23f640 1 -- start start 2024-09-17T03:14:23.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.745+0000 7f280890a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27fc036770 0x7f27fc0e6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.87:57890/0 (socket says 172.21.15.87:57890) 2024-09-17T03:14:23.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.745+0000 7f280890a640 1 -- 172.21.15.87:0/763701177 learned_addr learned my addr 172.21.15.87:0/763701177 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:23.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.745+0000 7f304ae69640 1 Processor -- start 2024-09-17T03:14:23.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.746+0000 7fb6c9dda640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb6bc008fa0 0x7fb6bc0d8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.746+0000 7fb6c9dda640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6bc03ae90 0x7fb6bc0d7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.746+0000 7fb6c9dda640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb6bc0d77d0 0x7fb6bc17a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.746+0000 7f304ae69640 1 -- start start 2024-09-17T03:14:23.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.746+0000 7f328d0b6640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32880ce7c0 0x7f32881619b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.746+0000 7fd02e6cc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd020035220 0x7fd020179480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.746+0000 7f328d0b6640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f328815e9f0 0x7f32880b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.746+0000 7f328d0b6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32880b4e10 0x7f32880b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.746+0000 7f328d0b6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f32880af500 con 0x7f32880b4e10 2024-09-17T03:14:23.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.746+0000 7f328d0b6640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f32880af380 con 0x7f32880ce7c0 2024-09-17T03:14:23.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.746+0000 7f328d0b6640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f32880af680 con 0x7f328815e9f0 2024-09-17T03:14:23.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.746+0000 7fb6c2ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6bc03ae90 0x7fb6bc0d7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.746+0000 7fb6c2ffd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6bc03ae90 0x7fb6bc0d7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.87:57906/0 (socket says 172.21.15.87:57906) 2024-09-17T03:14:23.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.746+0000 7fd02e6cc640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd02003af10 0x7fd02004df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.746+0000 7fd027fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd020035220 0x7fd020179480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.746+0000 7f328659c640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f328815e9f0 0x7f32880b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.746+0000 7fd027fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd020035220 0x7fd020179480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40190/0 (socket says 172.21.15.87:40190) 2024-09-17T03:14:23.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7fd027fff640 1 -- 172.21.15.87:0/3334752054 learned_addr learned my addr 172.21.15.87:0/3334752054 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:23.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7fd02e6cc640 1 --2- 172.21.15.87:0/3334752054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0200dfe70 0x7fd02004d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.746+0000 7f280890a640 1 -- 172.21.15.87:0/763701177 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f27fc036cc0 msgr2=0x7f27fc181e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.746+0000 7f280890a640 1 --2- 172.21.15.87:0/763701177 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f27fc036cc0 0x7f27fc181e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.746+0000 7f280890a640 1 -- 172.21.15.87:0/763701177 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27fc055280 msgr2=0x7f27fc1823c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.746+0000 7f280890a640 1 --2- 172.21.15.87:0/763701177 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27fc055280 0x7f27fc1823c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.746+0000 7f280890a640 1 -- 172.21.15.87:0/763701177 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27fc182ae0 con 0x7f27fc036770 2024-09-17T03:14:23.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7f280890a640 1 --2- 172.21.15.87:0/763701177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27fc036770 0x7f27fc0e6dd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f27f8006010 tx=0x7f27f8003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7f328659c640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f328815e9f0 0x7f32880b48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.87:50538/0 (socket says 172.21.15.87:50538) 2024-09-17T03:14:23.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7f328659c640 1 -- 172.21.15.87:0/3058792644 learned_addr learned my addr 172.21.15.87:0/3058792644 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:23.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7f328759e640 1 --2- 172.21.15.87:0/3058792644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32880b4e10 0x7f32880b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7fb6c9dda640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb6bc053b10 con 0x7fb6bc0d77d0 2024-09-17T03:14:23.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7fb6c2ffd640 1 -- 172.21.15.87:0/2912586481 learned_addr learned my addr 172.21.15.87:0/2912586481 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:23.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7f28017fa640 1 -- 172.21.15.87:0/763701177 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f803e030 con 0x7f27fc036770 2024-09-17T03:14:23.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7f28017fa640 1 -- 172.21.15.87:0/763701177 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27f8003d10 con 0x7f27fc036770 2024-09-17T03:14:23.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7fd02e6cc640 1 -- 172.21.15.87:0/3334752054 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd020053580 con 0x7fd020035220 2024-09-17T03:14:23.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7fb6c3fff640 1 --2- 172.21.15.87:0/2912586481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb6bc0d77d0 0x7fb6bc17a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7fb6c9dda640 1 -- 172.21.15.87:0/2912586481 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb6bc053990 con 0x7fb6bc03ae90 2024-09-17T03:14:23.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7fd02e6cc640 1 -- 172.21.15.87:0/3334752054 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd020053400 con 0x7fd0200dfe70 2024-09-17T03:14:23.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7fb6c9dda640 1 -- 172.21.15.87:0/2912586481 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb6bc053c90 con 0x7fb6bc008fa0 2024-09-17T03:14:23.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7f4f8292b640 1 Processor -- start 2024-09-17T03:14:23.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7fd02cc69640 1 --2- 172.21.15.87:0/3334752054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0200dfe70 0x7fd02004d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7fd02e6cc640 1 -- 172.21.15.87:0/3334752054 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd020053700 con 0x7fd02003af10 2024-09-17T03:14:23.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7f28017fa640 1 -- 172.21.15.87:0/763701177 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f8035040 con 0x7f27fc036770 2024-09-17T03:14:23.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7f280990c640 1 -- 172.21.15.87:0/763701177 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27fc0dc290 con 0x7f27fc036770 2024-09-17T03:14:23.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7f280990c640 1 -- 172.21.15.87:0/763701177 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27fc0d4f10 con 0x7f27fc036770 2024-09-17T03:14:23.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7fe025e8b640 1 Processor -- start 2024-09-17T03:14:23.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7f328659c640 1 -- 172.21.15.87:0/3058792644 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32880ce7c0 msgr2=0x7f32881619b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7f4f8292b640 1 -- start start 2024-09-17T03:14:23.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7f328659c640 1 --2- 172.21.15.87:0/3058792644 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f32880ce7c0 0x7f32881619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7fe025e8b640 1 -- start start 2024-09-17T03:14:23.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.748+0000 7f328659c640 1 -- 172.21.15.87:0/3058792644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32880b4e10 msgr2=0x7f32880b5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.747+0000 7fb6c37fe640 1 --2- 172.21.15.87:0/2912586481 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb6bc008fa0 0x7fb6bc0d8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.748+0000 7f114c23f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f113c005c60 0x7f113c0de6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.748+0000 7fd027fff640 1 -- 172.21.15.87:0/3334752054 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd02003af10 msgr2=0x7fd02004df00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T03:14:23.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.748+0000 7fd0277fe640 1 --2- 172.21.15.87:0/3334752054 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd02003af10 0x7fd02004df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.748+0000 7fb6c2ffd640 1 -- 172.21.15.87:0/2912586481 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb6bc008fa0 msgr2=0x7fb6bc0d8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.748+0000 7fb6c2ffd640 1 --2- 172.21.15.87:0/2912586481 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb6bc008fa0 0x7fb6bc0d8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.748+0000 7fb6c2ffd640 1 -- 172.21.15.87:0/2912586481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb6bc0d77d0 msgr2=0x7fb6bc17a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.748+0000 7fb6c2ffd640 1 --2- 172.21.15.87:0/2912586481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb6bc0d77d0 0x7fb6bc17a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.748+0000 7fb6c2ffd640 1 -- 172.21.15.87:0/2912586481 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6bc00c1f0 con 0x7fb6bc03ae90 2024-09-17T03:14:23.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.748+0000 7f304ae69640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3044070e00 0x7f304414bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.748+0000 7f304ae69640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f304415aed0 0x7f3044167d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.748+0000 7fd027fff640 1 --2- 172.21.15.87:0/3334752054 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd02003af10 0x7fd02004df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.748+0000 7fd027fff640 1 -- 172.21.15.87:0/3334752054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0200dfe70 msgr2=0x7fd02004d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.748+0000 7fd027fff640 1 --2- 172.21.15.87:0/3334752054 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0200dfe70 0x7fd02004d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.748+0000 7fd027fff640 1 -- 172.21.15.87:0/3334752054 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd02004e440 con 0x7fd020035220 2024-09-17T03:14:23.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.748+0000 7fd0277fe640 1 --2- 172.21.15.87:0/3334752054 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd02003af10 0x7fd02004df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:14:23.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.748+0000 7f280990c640 1 -- 172.21.15.87:0/763701177 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f27e4001030 con 0x7f27fc036770 2024-09-17T03:14:23.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.748+0000 7f328659c640 1 --2- 172.21.15.87:0/3058792644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32880b4e10 0x7f32880b5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.748+0000 7f328659c640 1 -- 172.21.15.87:0/3058792644 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3288157be0 con 0x7f328815e9f0 2024-09-17T03:14:23.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.748+0000 7f3048c05640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3044070e00 0x7f304414bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.748+0000 7f3048c05640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3044070e00 0x7f304414bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.87:50562/0 (socket says 172.21.15.87:50562) 2024-09-17T03:14:23.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.748+0000 7f3043fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f304415aed0 0x7f3044167d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.748+0000 7f3043fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f304415aed0 0x7f3044167d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40218/0 (socket says 172.21.15.87:40218) 2024-09-17T03:14:23.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.748+0000 7f3048c05640 1 -- 172.21.15.87:0/2485173823 learned_addr learned my addr 172.21.15.87:0/2485173823 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:23.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f114c23f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f113c036970 0x7f113c043350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f114c23f640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f113c0352b0 0x7f113c043890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f304ae69640 1 --2- 172.21.15.87:0/2485173823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f304415b650 0x7f30441682b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f304ae69640 1 -- 172.21.15.87:0/2485173823 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f30440d0750 con 0x7f304415aed0 2024-09-17T03:14:23.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f304ae69640 1 -- 172.21.15.87:0/2485173823 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f30440d05d0 con 0x7f304415b650 2024-09-17T03:14:23.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f304ae69640 1 -- 172.21.15.87:0/2485173823 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f30440d08d0 con 0x7f3044070e00 2024-09-17T03:14:23.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f114c23f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f113c01ce40 con 0x7f113c036970 2024-09-17T03:14:23.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f114c23f640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f113c01ccc0 con 0x7f113c005c60 2024-09-17T03:14:23.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f114c23f640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f113c01cfc0 con 0x7f113c0352b0 2024-09-17T03:14:23.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f328759e640 1 --2- 172.21.15.87:0/3058792644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32880b4e10 0x7f32880b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:14:23.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7fd027fff640 1 --2- 172.21.15.87:0/3334752054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd020035220 0x7fd020179480 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd028066830 tx=0x7fd02806a0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f11497da640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f113c036970 0x7f113c043350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f11497da640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f113c036970 0x7f113c043350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40234/0 (socket says 172.21.15.87:40234) 2024-09-17T03:14:23.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f11497da640 1 -- 172.21.15.87:0/2304947979 learned_addr learned my addr 172.21.15.87:0/2304947979 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:23.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f1149fdb640 1 --2- 172.21.15.87:0/2304947979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f113c005c60 0x7f113c0de6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f3049406640 1 --2- 172.21.15.87:0/2485173823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f304415b650 0x7f30441682b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.776 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f114a7dc640 1 --2- 172.21.15.87:0/2304947979 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f113c0352b0 0x7f113c043890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f3043fff640 1 -- 172.21.15.87:0/2485173823 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3044070e00 msgr2=0x7f304414bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f3043fff640 1 --2- 172.21.15.87:0/2485173823 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3044070e00 0x7f304414bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f3043fff640 1 -- 172.21.15.87:0/2485173823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f304415b650 msgr2=0x7f30441682b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f3043fff640 1 --2- 172.21.15.87:0/2485173823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f304415b650 0x7f30441682b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f3043fff640 1 -- 172.21.15.87:0/2485173823 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30440b54e0 con 0x7f304415aed0 2024-09-17T03:14:23.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7fd024ff9640 1 -- 172.21.15.87:0/3334752054 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd028098070 con 0x7fd020035220 2024-09-17T03:14:23.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7fd024ff9640 1 -- 172.21.15.87:0/3334752054 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0280729c0 con 0x7fd020035220 2024-09-17T03:14:23.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.750+0000 7fd02e6cc640 1 -- 172.21.15.87:0/3334752054 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd020179d10 con 0x7fd020035220 2024-09-17T03:14:23.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.750+0000 7f11497da640 1 -- 172.21.15.87:0/2304947979 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f113c0352b0 msgr2=0x7f113c043890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.750+0000 7f3043fff640 1 --2- 172.21.15.87:0/2485173823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f304415aed0 0x7f3044167d70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f303c031000 tx=0x7f303c003940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.750+0000 7fe025e8b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0200b3c30 0x7fe0200b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.750+0000 7f4f8292b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f6c0048b0 0x7f4f74175f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.750+0000 7f4f8292b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f740321b0 0x7f4f740de630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.750+0000 7f30417fa640 1 -- 172.21.15.87:0/2485173823 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c03e030 con 0x7f304415aed0 2024-09-17T03:14:23.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.750+0000 7f30417fa640 1 -- 172.21.15.87:0/2485173823 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f303c0047a0 con 0x7f304415aed0 2024-09-17T03:14:23.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.750+0000 7f30417fa640 1 -- 172.21.15.87:0/2485173823 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303c035040 con 0x7f304415aed0 2024-09-17T03:14:23.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.750+0000 7f11497da640 1 --2- 172.21.15.87:0/2304947979 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f113c0352b0 0x7f113c043890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.750+0000 7f11497da640 1 -- 172.21.15.87:0/2304947979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f113c005c60 msgr2=0x7f113c0de6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.750+0000 7f11497da640 1 --2- 172.21.15.87:0/2304947979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f113c005c60 0x7f113c0de6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.750+0000 7f4f7b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f740321b0 0x7f4f740de630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.750+0000 7f11497da640 1 -- 172.21.15.87:0/2304947979 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f113c0461e0 con 0x7f113c036970 2024-09-17T03:14:23.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.750+0000 7f4f7b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f740321b0 0x7f4f740de630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40238/0 (socket says 172.21.15.87:40238) 2024-09-17T03:14:23.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.750+0000 7f1149fdb640 1 --2- 172.21.15.87:0/2304947979 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f113c005c60 0x7f113c0de6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:14:23.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.750+0000 7f114a7dc640 1 --2- 172.21.15.87:0/2304947979 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f113c0352b0 0x7f113c043890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:14:23.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.750+0000 7f4f7bfff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f6c0048b0 0x7f4f74175f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.750+0000 7f4f7bfff640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f6c0048b0 0x7f4f74175f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.87:57946/0 (socket says 172.21.15.87:57946) 2024-09-17T03:14:23.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.750+0000 7f4f7b7fe640 1 -- 172.21.15.87:0/4110884642 learned_addr learned my addr 172.21.15.87:0/4110884642 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:23.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.750+0000 7f4f8292b640 1 --2- 172.21.15.87:0/4110884642 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4f74020360 0x7f4f74020770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.750+0000 7f4f8292b640 1 -- 172.21.15.87:0/4110884642 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7400bdb0 con 0x7f4f740321b0 2024-09-17T03:14:23.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.751+0000 7f4f8292b640 1 -- 172.21.15.87:0/4110884642 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7400bc30 con 0x7f4f6c0048b0 2024-09-17T03:14:23.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.751+0000 7f4f8292b640 1 -- 172.21.15.87:0/4110884642 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4f740df180 con 0x7f4f74020360 2024-09-17T03:14:23.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.751+0000 7f4f80ec8640 1 --2- 172.21.15.87:0/4110884642 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4f74020360 0x7f4f74020770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.750+0000 7fd024ff9640 1 -- 172.21.15.87:0/3334752054 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd028072d90 con 0x7fd020035220 2024-09-17T03:14:23.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.751+0000 7fd02e6cc640 1 -- 172.21.15.87:0/3334752054 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd02017a2d0 con 0x7fd020035220 2024-09-17T03:14:23.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.751+0000 7f4f80ec8640 1 -- 172.21.15.87:0/4110884642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f6c0048b0 msgr2=0x7f4f74175f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f328659c640 1 --2- 172.21.15.87:0/3058792644 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f328815e9f0 0x7f32880b48d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f327800c430 tx=0x7f327800c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f326f7fe640 1 -- 172.21.15.87:0/3058792644 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278017070 con 0x7f328815e9f0 2024-09-17T03:14:23.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f326f7fe640 1 -- 172.21.15.87:0/3058792644 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32780053c0 con 0x7f328815e9f0 2024-09-17T03:14:23.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f326f7fe640 1 -- 172.21.15.87:0/3058792644 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278003850 con 0x7f328815e9f0 2024-09-17T03:14:23.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f328d0b6640 1 -- 172.21.15.87:0/3058792644 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3288159d20 con 0x7f328815e9f0 2024-09-17T03:14:23.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7f328d0b6640 1 -- 172.21.15.87:0/3058792644 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3288157950 con 0x7f328815e9f0 2024-09-17T03:14:23.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.748+0000 7fb6c3fff640 1 --2- 172.21.15.87:0/2912586481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb6bc0d77d0 0x7fb6bc17a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:14:23.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.749+0000 7fb6c2ffd640 1 --2- 172.21.15.87:0/2912586481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6bc03ae90 0x7fb6bc0d7290 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8000c00 tx=0x7fb6b8033ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.751+0000 7f4f80ec8640 1 --2- 172.21.15.87:0/4110884642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4f6c0048b0 0x7f4f74175f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.751+0000 7f28017fa640 1 -- 172.21.15.87:0/763701177 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f27f8041030 con 0x7f27fc036770 2024-09-17T03:14:23.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.751+0000 7f11497da640 1 --2- 172.21.15.87:0/2304947979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f113c036970 0x7f113c043350 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f113400d740 tx=0x7f113400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.751+0000 7f4f80ec8640 1 -- 172.21.15.87:0/4110884642 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f740321b0 msgr2=0x7f4f740de630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.751+0000 7f4f80ec8640 1 --2- 172.21.15.87:0/4110884642 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f740321b0 0x7f4f740de630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.752+0000 7f304ae69640 1 -- 172.21.15.87:0/2485173823 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30440c7190 con 0x7f304415aed0 2024-09-17T03:14:23.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.752+0000 7f113a7fc640 1 -- 172.21.15.87:0/2304947979 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134017070 con 0x7f113c036970 2024-09-17T03:14:23.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.752+0000 7f304ae69640 1 -- 172.21.15.87:0/2485173823 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30440b4f20 con 0x7f304415aed0 2024-09-17T03:14:23.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.751+0000 7f4f80ec8640 1 -- 172.21.15.87:0/4110884642 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f740e29f0 con 0x7f4f74020360 2024-09-17T03:14:23.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.751+0000 7f4f7b7fe640 1 --2- 172.21.15.87:0/4110884642 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4f740321b0 0x7f4f740de630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:14:23.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.752+0000 7f114c23f640 1 -- 172.21.15.87:0/2304947979 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f113c0464c0 con 0x7f113c036970 2024-09-17T03:14:23.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.752+0000 7f114c23f640 1 -- 172.21.15.87:0/2304947979 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f113c022170 con 0x7f113c036970 2024-09-17T03:14:23.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.752+0000 7f4f80ec8640 1 --2- 172.21.15.87:0/4110884642 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4f74020360 0x7f4f74020770 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c051d10 tx=0x7f4f7c06a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.752+0000 7f113a7fc640 1 -- 172.21.15.87:0/2304947979 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1134009c30 con 0x7f113c036970 2024-09-17T03:14:23.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.752+0000 7f28017fa640 1 --2- 172.21.15.87:0/763701177 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f27e8078b90 0x7f27e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.752+0000 7f28017fa640 1 -- 172.21.15.87:0/763701177 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f27f80bb320 con 0x7f27fc036770 2024-09-17T03:14:23.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.752+0000 7f4f78ff9640 1 -- 172.21.15.87:0/4110884642 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c098070 con 0x7f4f74020360 2024-09-17T03:14:23.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.752+0000 7fd024ff9640 1 -- 172.21.15.87:0/3334752054 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd02809b070 con 0x7fd020035220 2024-09-17T03:14:23.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.753+0000 7f4f78ff9640 1 -- 172.21.15.87:0/4110884642 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f7c072930 con 0x7f4f74020360 2024-09-17T03:14:23.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.753+0000 7f113a7fc640 1 -- 172.21.15.87:0/2304947979 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11340106c0 con 0x7f113c036970 2024-09-17T03:14:23.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.752+0000 7f2803fff640 1 --2- 172.21.15.87:0/763701177 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f27e8078b90 0x7f27e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.752+0000 7f28017fa640 1 --2- 172.21.15.87:0/763701177 >> [v2:172.21.15.87:6808/3467605435,v1:172.21.15.87:6809/3467605435] conn(0x7f27e8080e60 0x7f27e80832b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.752+0000 7f28017fa640 1 -- 172.21.15.87:0/763701177 --> [v2:172.21.15.87:6808/3467605435,v1:172.21.15.87:6809/3467605435] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f27f8005310 con 0x7f27e8080e60 2024-09-17T03:14:23.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.752+0000 7f28017fa640 1 -- 172.21.15.87:0/763701177 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f27f80c2050 con 0x7f27fc036770 2024-09-17T03:14:23.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.753+0000 7f280910b640 1 --2- 172.21.15.87:0/763701177 >> [v2:172.21.15.87:6808/3467605435,v1:172.21.15.87:6809/3467605435] conn(0x7f27e8080e60 0x7f27e80832b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.753+0000 7f4f8292b640 1 -- 172.21.15.87:0/4110884642 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f74020cb0 con 0x7f4f74020360 2024-09-17T03:14:23.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.753+0000 7f114c23f640 1 -- 172.21.15.87:0/2304947979 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f112c001030 con 0x7f113c036970 2024-09-17T03:14:23.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.753+0000 7f4f8292b640 1 -- 172.21.15.87:0/4110884642 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f740e2710 con 0x7f4f74020360 2024-09-17T03:14:23.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.753+0000 7fd02e6cc640 1 -- 172.21.15.87:0/3334752054 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd020023550 con 0x7fd020035220 2024-09-17T03:14:23.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.753+0000 7f280910b640 1 --2- 172.21.15.87:0/763701177 >> [v2:172.21.15.87:6808/3467605435,v1:172.21.15.87:6809/3467605435] conn(0x7f27e8080e60 0x7f27e80832b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.753+0000 7fd024ff9640 1 --2- 172.21.15.87:0/3334752054 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd00c078c60 0x7fd00c07b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.753+0000 7fe01f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0200b3c30 0x7fe0200b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.753+0000 7fe025e8b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0200b9770 0x7fe0201f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.754+0000 7fd024ff9640 1 -- 172.21.15.87:0/3334752054 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd028116990 con 0x7fd020035220 2024-09-17T03:14:23.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.754+0000 7fe01f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0200b3c30 0x7fe0200b1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40236/0 (socket says 172.21.15.87:40236) 2024-09-17T03:14:23.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.754+0000 7fe01f7fe640 1 -- 172.21.15.87:0/3195857615 learned_addr learned my addr 172.21.15.87:0/3195857615 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:23.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.754+0000 7fe025e8b640 1 --2- 172.21.15.87:0/3195857615 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe0200cfca0 0x7fe0201f93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.754+0000 7fe025e8b640 1 -- 172.21.15.87:0/3195857615 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe0200d1220 con 0x7fe0200b3c30 2024-09-17T03:14:23.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.754+0000 7fd024ff9640 1 --2- 172.21.15.87:0/3334752054 >> [v2:172.21.15.87:6824/2430409749,v1:172.21.15.87:6825/2430409749] conn(0x7fd00c080e50 0x7fd00c0832a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.754+0000 7f2803fff640 1 --2- 172.21.15.87:0/763701177 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f27e8078b90 0x7f27e807b0a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f27fc039e60 tx=0x7f27f0005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.750+0000 7fb6affff640 1 -- 172.21.15.87:0/2912586481 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8031d60 con 0x7fb6bc03ae90 2024-09-17T03:14:23.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.750+0000 7fb6affff640 1 -- 172.21.15.87:0/2912586481 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6b8033070 con 0x7fb6bc03ae90 2024-09-17T03:14:23.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.751+0000 7fb6affff640 1 -- 172.21.15.87:0/2912586481 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b80334c0 con 0x7fb6bc03ae90 2024-09-17T03:14:23.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.754+0000 7fb6c9dda640 1 -- 172.21.15.87:0/2912586481 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6bc00c4d0 con 0x7fb6bc03ae90 2024-09-17T03:14:23.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.754+0000 7f28017fa640 1 -- 172.21.15.87:0/763701177 <== osd.1 v2:172.21.15.87:6808/3467605435 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f27f8005310 con 0x7f27e8080e60 2024-09-17T03:14:23.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.754+0000 7fb6c9dda640 1 -- 172.21.15.87:0/2912586481 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6bc00ca90 con 0x7fb6bc03ae90 2024-09-17T03:14:23.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.755+0000 7f4f78ff9640 1 -- 172.21.15.87:0/4110884642 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c076e60 con 0x7f4f74020360 2024-09-17T03:14:23.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.754+0000 7fd0277fe640 1 --2- 172.21.15.87:0/3334752054 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd00c078c60 0x7fd00c07b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.755+0000 7fd024ff9640 1 -- 172.21.15.87:0/3334752054 --> [v2:172.21.15.87:6824/2430409749,v1:172.21.15.87:6825/2430409749] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd02806fea0 con 0x7fd00c080e50 2024-09-17T03:14:23.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.755+0000 7fd024ff9640 1 -- 172.21.15.87:0/3334752054 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd02811d050 con 0x7fd020035220 2024-09-17T03:14:23.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.755+0000 7fd02cc69640 1 --2- 172.21.15.87:0/3334752054 >> [v2:172.21.15.87:6824/2430409749,v1:172.21.15.87:6825/2430409749] conn(0x7fd00c080e50 0x7fd00c0832a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.755+0000 7fd0277fe640 1 --2- 172.21.15.87:0/3334752054 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd00c078c60 0x7fd00c07b170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd0200326a0 tx=0x7fd01c008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.755+0000 7f4f78ff9640 1 -- 172.21.15.87:0/4110884642 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4f7c09b070 con 0x7f4f74020360 2024-09-17T03:14:23.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.756+0000 7fb6c9dda640 1 -- 172.21.15.87:0/2912586481 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb6bc17a0a0 con 0x7fb6bc03ae90 2024-09-17T03:14:23.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.756+0000 7f4f8292b640 1 -- 172.21.15.87:0/4110884642 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4f74054c70 con 0x7f4f74020360 2024-09-17T03:14:23.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.756+0000 7f113a7fc640 1 -- 172.21.15.87:0/2304947979 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f113401a070 con 0x7f113c036970 2024-09-17T03:14:23.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.754+0000 7fe01effd640 1 --2- 172.21.15.87:0/3195857615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0200b9770 0x7fe0201f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.754+0000 7fe01ffff640 1 --2- 172.21.15.87:0/3195857615 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe0200cfca0 0x7fe0201f93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.756+0000 7fe025e8b640 1 -- 172.21.15.87:0/3195857615 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe0200d10a0 con 0x7fe0200b9770 2024-09-17T03:14:23.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.756+0000 7fe025e8b640 1 -- 172.21.15.87:0/3195857615 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe0200d13a0 con 0x7fe0200cfca0 2024-09-17T03:14:23.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.752+0000 7f328d0b6640 1 -- 172.21.15.87:0/3058792644 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f328815a6c0 con 0x7f328815e9f0 2024-09-17T03:14:23.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.754+0000 7f326f7fe640 1 -- 172.21.15.87:0/3058792644 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f327801a070 con 0x7f328815e9f0 2024-09-17T03:14:23.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.755+0000 7f326f7fe640 1 --2- 172.21.15.87:0/3058792644 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3274078b90 0x7f327407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.755+0000 7f3286d9d640 1 --2- 172.21.15.87:0/3058792644 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3274078b90 0x7f327407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.756+0000 7f326f7fe640 1 -- 172.21.15.87:0/3058792644 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f3278098780 con 0x7f328815e9f0 2024-09-17T03:14:23.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.756+0000 7f326f7fe640 1 --2- 172.21.15.87:0/3058792644 >> [v2:172.21.15.149:6826/2182999530,v1:172.21.15.149:6827/2182999530] conn(0x7f3274081990 0x7f3274083e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.756+0000 7f326f7fe640 1 -- 172.21.15.87:0/3058792644 --> [v2:172.21.15.149:6826/2182999530,v1:172.21.15.149:6827/2182999530] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3274084550 con 0x7f3274081990 2024-09-17T03:14:23.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.757+0000 7f4f78ff9640 1 --2- 172.21.15.87:0/4110884642 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4f60078c60 0x7f4f6007b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.756+0000 7f326f7fe640 1 -- 172.21.15.87:0/3058792644 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f327805e670 con 0x7f328815e9f0 2024-09-17T03:14:23.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.757+0000 7fb6affff640 1 -- 172.21.15.87:0/2912586481 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb6b8033660 con 0x7fb6bc03ae90 2024-09-17T03:14:23.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.757+0000 7f4f7bfff640 1 --2- 172.21.15.87:0/4110884642 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4f60078c60 0x7f4f6007b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.757+0000 7f4f78ff9640 1 -- 172.21.15.87:0/4110884642 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4f7c116790 con 0x7f4f74020360 2024-09-17T03:14:23.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.757+0000 7f113a7fc640 1 --2- 172.21.15.87:0/2304947979 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1128078b90 0x7f112807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.756+0000 7fd02cc69640 1 --2- 172.21.15.87:0/3334752054 >> [v2:172.21.15.87:6824/2430409749,v1:172.21.15.87:6825/2430409749] conn(0x7fd00c080e50 0x7fd00c0832a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.757+0000 7f328759e640 1 --2- 172.21.15.87:0/3058792644 >> [v2:172.21.15.149:6826/2182999530,v1:172.21.15.149:6827/2182999530] conn(0x7f3274081990 0x7f3274083e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.757+0000 7f4f78ff9640 1 --2- 172.21.15.87:0/4110884642 >> [v2:172.21.15.149:6802/2414907089,v1:172.21.15.149:6803/2414907089] conn(0x7f4f60080ed0 0x7f4f60083320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.757+0000 7f1149fdb640 1 --2- 172.21.15.87:0/2304947979 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1128078b90 0x7f112807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.757+0000 7f4f78ff9640 1 -- 172.21.15.87:0/4110884642 --> [v2:172.21.15.149:6802/2414907089,v1:172.21.15.149:6803/2414907089] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4f7c072930 con 0x7f4f60080ed0 2024-09-17T03:14:23.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.757+0000 7fe01ffff640 1 -- 172.21.15.87:0/3195857615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0200b9770 msgr2=0x7fe0201f6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.757+0000 7fe01ffff640 1 --2- 172.21.15.87:0/3195857615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0200b9770 0x7fe0201f6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.757+0000 7fe01ffff640 1 -- 172.21.15.87:0/3195857615 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0200b3c30 msgr2=0x7fe0200b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.757+0000 7fe01ffff640 1 --2- 172.21.15.87:0/3195857615 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0200b3c30 0x7fe0200b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.757+0000 7fe01ffff640 1 -- 172.21.15.87:0/3195857615 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0200b15b0 con 0x7fe0200cfca0 2024-09-17T03:14:23.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.757+0000 7f113a7fc640 1 -- 172.21.15.87:0/2304947979 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f113409b160 con 0x7f113c036970 2024-09-17T03:14:23.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.757+0000 7f4f78ff9640 1 -- 172.21.15.87:0/4110884642 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4f7c11d050 con 0x7f4f74020360 2024-09-17T03:14:23.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.757+0000 7fd024ff9640 1 -- 172.21.15.87:0/3334752054 <== osd.0 v2:172.21.15.87:6824/2430409749 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd02806fea0 con 0x7fd00c080e50 2024-09-17T03:14:23.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.757+0000 7f4f7b7fe640 1 --2- 172.21.15.87:0/4110884642 >> [v2:172.21.15.149:6802/2414907089,v1:172.21.15.149:6803/2414907089] conn(0x7f4f60080ed0 0x7f4f60083320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.757+0000 7f113a7fc640 1 --2- 172.21.15.87:0/2304947979 >> [v2:172.21.15.87:6816/4253161216,v1:172.21.15.87:6817/4253161216] conn(0x7f1128080d80 0x7f11280831d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.757+0000 7f4f7bfff640 1 --2- 172.21.15.87:0/4110884642 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4f60078c60 0x7f4f6007b170 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4f700056e0 tx=0x7f4f70009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.756+0000 7f30417fa640 1 -- 172.21.15.87:0/2485173823 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f303c041030 con 0x7f304415aed0 2024-09-17T03:14:23.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.758+0000 7f30417fa640 1 --2- 172.21.15.87:0/2485173823 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3024078c90 0x7f302407b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.758+0000 7fe01ffff640 1 --2- 172.21.15.87:0/3195857615 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe0200cfca0 0x7fe0201f93a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe0200ba0e0 tx=0x7fe01400ea60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.758+0000 7f113a7fc640 1 -- 172.21.15.87:0/2304947979 --> [v2:172.21.15.87:6816/4253161216,v1:172.21.15.87:6817/4253161216] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1134026e50 con 0x7f1128080d80 2024-09-17T03:14:23.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.758+0000 7f113a7fc640 1 -- 172.21.15.87:0/2304947979 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f113409b5f0 con 0x7f113c036970 2024-09-17T03:14:23.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.758+0000 7fe007fff640 1 -- 172.21.15.87:0/3195857615 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe014004220 con 0x7fe0200cfca0 2024-09-17T03:14:23.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.758+0000 7f3048c05640 1 --2- 172.21.15.87:0/2485173823 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3024078c90 0x7f302407b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.758+0000 7fe007fff640 1 -- 172.21.15.87:0/3195857615 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe014005020 con 0x7fe0200cfca0 2024-09-17T03:14:23.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.758+0000 7f114a7dc640 1 --2- 172.21.15.87:0/2304947979 >> [v2:172.21.15.87:6816/4253161216,v1:172.21.15.87:6817/4253161216] conn(0x7f1128080d80 0x7f11280831d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.758+0000 7f4f7b7fe640 1 --2- 172.21.15.87:0/4110884642 >> [v2:172.21.15.149:6802/2414907089,v1:172.21.15.149:6803/2414907089] conn(0x7f4f60080ed0 0x7f4f60083320 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.758+0000 7f30417fa640 1 -- 172.21.15.87:0/2485173823 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f303c0bb460 con 0x7f304415aed0 2024-09-17T03:14:23.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.758+0000 7f1149fdb640 1 --2- 172.21.15.87:0/2304947979 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1128078b90 0x7f112807b0a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1140034000 tx=0x7f1140004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.758+0000 7f303affd640 1 --2- 172.21.15.87:0/2485173823 >> [v2:172.21.15.149:6818/2771010477,v1:172.21.15.149:6819/2771010477] conn(0x7f30440c7670 0x7f30440c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.757+0000 7f3286d9d640 1 --2- 172.21.15.87:0/3058792644 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3274078b90 0x7f327407b0a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f327c000f60 tx=0x7f327c0079b0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.758+0000 7f303affd640 1 -- 172.21.15.87:0/2485173823 --> [v2:172.21.15.149:6818/2771010477,v1:172.21.15.149:6819/2771010477] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f30440b49f0 con 0x7f30440c7670 2024-09-17T03:14:23.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.758+0000 7f328759e640 1 --2- 172.21.15.87:0/3058792644 >> [v2:172.21.15.149:6826/2182999530,v1:172.21.15.149:6827/2182999530] conn(0x7f3274081990 0x7f3274083e80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.758+0000 7fe007fff640 1 -- 172.21.15.87:0/3195857615 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe014005470 con 0x7fe0200cfca0 2024-09-17T03:14:23.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.758+0000 7fe025e8b640 1 -- 172.21.15.87:0/3195857615 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0201f5960 con 0x7fe0200cfca0 2024-09-17T03:14:23.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.758+0000 7f3049406640 1 --2- 172.21.15.87:0/2485173823 >> [v2:172.21.15.149:6818/2771010477,v1:172.21.15.149:6819/2771010477] conn(0x7f30440c7670 0x7f30440c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.759+0000 7fe025e8b640 1 -- 172.21.15.87:0/3195857615 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0200cd4e0 con 0x7fe0200cfca0 2024-09-17T03:14:23.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.759+0000 7f3048c05640 1 --2- 172.21.15.87:0/2485173823 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3024078c90 0x7f302407b1a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f302c009990 tx=0x7f302c009110 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.758+0000 7fb6affff640 1 --2- 172.21.15.87:0/2912586481 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb6a8078c90 0x7fb6a807b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.759+0000 7fb6c37fe640 1 --2- 172.21.15.87:0/2912586481 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb6a8078c90 0x7fb6a807b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.759+0000 7f114a7dc640 1 --2- 172.21.15.87:0/2304947979 >> [v2:172.21.15.87:6816/4253161216,v1:172.21.15.87:6817/4253161216] conn(0x7f1128080d80 0x7f11280831d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.759+0000 7f326f7fe640 1 -- 172.21.15.87:0/3058792644 <== osd.5 v2:172.21.15.149:6826/2182999530 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3274084550 con 0x7f3274081990 2024-09-17T03:14:23.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.759+0000 7f4f78ff9640 1 -- 172.21.15.87:0/4110884642 <== osd.4 v2:172.21.15.149:6802/2414907089 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4f7c072930 con 0x7f4f60080ed0 2024-09-17T03:14:23.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.759+0000 7fb6affff640 1 -- 172.21.15.87:0/2912586481 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb6b80bc000 con 0x7fb6bc03ae90 2024-09-17T03:14:23.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.759+0000 7fb6affff640 1 --2- 172.21.15.87:0/2912586481 >> [v2:172.21.15.87:6800/1450834766,v1:172.21.15.87:6801/1450834766] conn(0x7fb6a8080f60 0x7fb6a80833b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.759+0000 7fe025e8b640 1 -- 172.21.15.87:0/3195857615 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe0200d36f0 con 0x7fe0200cfca0 2024-09-17T03:14:23.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.760+0000 7fb6c37fe640 1 --2- 172.21.15.87:0/2912586481 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb6a8078c90 0x7fb6a807b1a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0006fd0 tx=0x7fb6b0008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.760+0000 7fb6affff640 1 -- 172.21.15.87:0/2912586481 --> [v2:172.21.15.87:6800/1450834766,v1:172.21.15.87:6801/1450834766] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb6b8037e90 con 0x7fb6a8080f60 2024-09-17T03:14:23.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.760+0000 7fb6affff640 1 -- 172.21.15.87:0/2912586481 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb6b80be600 con 0x7fb6bc03ae90 2024-09-17T03:14:23.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.759+0000 7f3049406640 1 --2- 172.21.15.87:0/2485173823 >> [v2:172.21.15.149:6818/2771010477,v1:172.21.15.149:6819/2771010477] conn(0x7f30440c7670 0x7f30440c7a80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.760+0000 7f113a7fc640 1 -- 172.21.15.87:0/2304947979 <== osd.3 v2:172.21.15.87:6816/4253161216 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1134026e50 con 0x7f1128080d80 2024-09-17T03:14:23.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.760+0000 7f30417fa640 1 -- 172.21.15.87:0/2485173823 <== osd.7 v2:172.21.15.149:6818/2771010477 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f30440b49f0 con 0x7f30440c7670 2024-09-17T03:14:23.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.761+0000 7fe007fff640 1 -- 172.21.15.87:0/3195857615 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe0140047a0 con 0x7fe0200cfca0 2024-09-17T03:14:23.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.762+0000 7fe007fff640 1 --2- 172.21.15.87:0/3195857615 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe00c078b90 0x7fe00c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.762+0000 7fb6c3fff640 1 --2- 172.21.15.87:0/2912586481 >> [v2:172.21.15.87:6800/1450834766,v1:172.21.15.87:6801/1450834766] conn(0x7fb6a8080f60 0x7fb6a80833b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.762+0000 7fe01f7fe640 1 --2- 172.21.15.87:0/3195857615 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe00c078b90 0x7fe00c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.762+0000 7fb6c3fff640 1 --2- 172.21.15.87:0/2912586481 >> [v2:172.21.15.87:6800/1450834766,v1:172.21.15.87:6801/1450834766] conn(0x7fb6a8080f60 0x7fb6a80833b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.763+0000 7fe007fff640 1 -- 172.21.15.87:0/3195857615 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe01409b140 con 0x7fe0200cfca0 2024-09-17T03:14:23.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.763+0000 7fe01f7fe640 1 --2- 172.21.15.87:0/3195857615 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe00c078b90 0x7fe00c07b0a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe010009590 tx=0x7fe010004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.763+0000 7fe007fff640 1 --2- 172.21.15.87:0/3195857615 >> [v2:172.21.15.149:6810/4289519146,v1:172.21.15.149:6811/4289519146] conn(0x7fe00c080e60 0x7fe00c0832b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:23.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.763+0000 7fe007fff640 1 -- 172.21.15.87:0/3195857615 --> [v2:172.21.15.149:6810/4289519146,v1:172.21.15.149:6811/4289519146] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe00c083980 con 0x7fe00c080e60 2024-09-17T03:14:23.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.763+0000 7fb6affff640 1 -- 172.21.15.87:0/2912586481 <== osd.2 v2:172.21.15.87:6800/1450834766 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb6b8037e90 con 0x7fb6a8080f60 2024-09-17T03:14:23.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.763+0000 7fe007fff640 1 -- 172.21.15.87:0/3195857615 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe0140a2050 con 0x7fe0200cfca0 2024-09-17T03:14:23.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.764+0000 7fe01effd640 1 --2- 172.21.15.87:0/3195857615 >> [v2:172.21.15.149:6810/4289519146,v1:172.21.15.149:6811/4289519146] conn(0x7fe00c080e60 0x7fe00c0832b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:23.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.764+0000 7fe01effd640 1 --2- 172.21.15.87:0/3195857615 >> [v2:172.21.15.149:6810/4289519146,v1:172.21.15.149:6811/4289519146] conn(0x7fe00c080e60 0x7fe00c0832b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:23.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.765+0000 7fe007fff640 1 -- 172.21.15.87:0/3195857615 <== osd.6 v2:172.21.15.149:6810/4289519146 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe00c083980 con 0x7fe00c080e60 2024-09-17T03:14:23.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.785+0000 7f280990c640 1 -- 172.21.15.87:0/763701177 --> [v2:172.21.15.87:6808/3467605435,v1:172.21.15.87:6809/3467605435] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f27e4002dd0 con 0x7f27e8080e60 2024-09-17T03:14:23.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.785+0000 7f28017fa640 1 -- 172.21.15.87:0/763701177 <== osd.1 v2:172.21.15.87:6808/3467605435 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f27e4002dd0 con 0x7f27e8080e60 2024-09-17T03:14:23.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.786+0000 7f27eeffd640 1 -- 172.21.15.87:0/763701177 >> [v2:172.21.15.87:6808/3467605435,v1:172.21.15.87:6809/3467605435] conn(0x7f27e8080e60 msgr2=0x7f27e80832b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.786+0000 7f27eeffd640 1 --2- 172.21.15.87:0/763701177 >> [v2:172.21.15.87:6808/3467605435,v1:172.21.15.87:6809/3467605435] conn(0x7f27e8080e60 0x7f27e80832b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.786+0000 7f4f8292b640 1 -- 172.21.15.87:0/4110884642 --> [v2:172.21.15.149:6802/2414907089,v1:172.21.15.149:6803/2414907089] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4f74054990 con 0x7f4f60080ed0 2024-09-17T03:14:23.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.787+0000 7f27eeffd640 1 -- 172.21.15.87:0/763701177 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f27e8078b90 msgr2=0x7f27e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.787+0000 7f27eeffd640 1 --2- 172.21.15.87:0/763701177 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f27e8078b90 0x7f27e807b0a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f27fc039e60 tx=0x7f27f0005f90 comp rx=0 tx=0).stop 2024-09-17T03:14:23.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.787+0000 7f27eeffd640 1 -- 172.21.15.87:0/763701177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27fc036770 msgr2=0x7f27fc0e6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.787+0000 7f27eeffd640 1 --2- 172.21.15.87:0/763701177 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27fc036770 0x7f27fc0e6dd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f27f8006010 tx=0x7f27f8003820 comp rx=0 tx=0).stop 2024-09-17T03:14:23.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.787+0000 7f280910b640 1 -- 172.21.15.87:0/763701177 reap_dead start 2024-09-17T03:14:23.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.787+0000 7f27eeffd640 1 -- 172.21.15.87:0/763701177 shutdown_connections 2024-09-17T03:14:23.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.787+0000 7f4f78ff9640 1 -- 172.21.15.87:0/4110884642 <== osd.4 v2:172.21.15.149:6802/2414907089 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4f74054990 con 0x7f4f60080ed0 2024-09-17T03:14:23.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.787+0000 7f27eeffd640 1 -- 172.21.15.87:0/763701177 >> 172.21.15.87:0/763701177 conn(0x7f27fc01eae0 msgr2=0x7f27fc0338a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:23.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.788+0000 7f27eeffd640 1 -- 172.21.15.87:0/763701177 shutdown_connections 2024-09-17T03:14:23.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.788+0000 7f27eeffd640 1 -- 172.21.15.87:0/763701177 wait complete. 2024-09-17T03:14:23.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.788+0000 7f4f667fc640 1 -- 172.21.15.87:0/4110884642 >> [v2:172.21.15.149:6802/2414907089,v1:172.21.15.149:6803/2414907089] conn(0x7f4f60080ed0 msgr2=0x7f4f60083320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.788+0000 7f4f667fc640 1 --2- 172.21.15.87:0/4110884642 >> [v2:172.21.15.149:6802/2414907089,v1:172.21.15.149:6803/2414907089] conn(0x7f4f60080ed0 0x7f4f60083320 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.788+0000 7f4f667fc640 1 -- 172.21.15.87:0/4110884642 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4f60078c60 msgr2=0x7f4f6007b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.788+0000 7fd02e6cc640 1 -- 172.21.15.87:0/3334752054 --> [v2:172.21.15.87:6824/2430409749,v1:172.21.15.87:6825/2430409749] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd020052f40 con 0x7fd00c080e50 2024-09-17T03:14:23.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.788+0000 7f328d0b6640 1 -- 172.21.15.87:0/3058792644 --> [v2:172.21.15.149:6826/2182999530,v1:172.21.15.149:6827/2182999530] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f32880aec30 con 0x7f3274081990 2024-09-17T03:14:23.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.788+0000 7f4f667fc640 1 --2- 172.21.15.87:0/4110884642 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4f60078c60 0x7f4f6007b170 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4f700056e0 tx=0x7f4f70009040 comp rx=0 tx=0).stop 2024-09-17T03:14:23.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.788+0000 7fd024ff9640 1 -- 172.21.15.87:0/3334752054 <== osd.0 v2:172.21.15.87:6824/2430409749 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd020052f40 con 0x7fd00c080e50 2024-09-17T03:14:23.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.789+0000 7f4f667fc640 1 -- 172.21.15.87:0/4110884642 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4f74020360 msgr2=0x7f4f74020770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.789+0000 7f4f667fc640 1 --2- 172.21.15.87:0/4110884642 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4f74020360 0x7f4f74020770 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c051d10 tx=0x7f4f7c06a1a0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.789+0000 7f4f80ec8640 1 -- 172.21.15.87:0/4110884642 reap_dead start 2024-09-17T03:14:23.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.789+0000 7f4f667fc640 1 -- 172.21.15.87:0/4110884642 shutdown_connections 2024-09-17T03:14:23.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.789+0000 7f4f667fc640 1 -- 172.21.15.87:0/4110884642 >> 172.21.15.87:0/4110884642 conn(0x7f4f74051420 msgr2=0x7f4f74004a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:23.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.789+0000 7f4f667fc640 1 -- 172.21.15.87:0/4110884642 shutdown_connections 2024-09-17T03:14:23.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.789+0000 7f4f667fc640 1 -- 172.21.15.87:0/4110884642 wait complete. 2024-09-17T03:14:23.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.789+0000 7fd02e6cc640 1 -- 172.21.15.87:0/3334752054 >> [v2:172.21.15.87:6824/2430409749,v1:172.21.15.87:6825/2430409749] conn(0x7fd00c080e50 msgr2=0x7fd00c0832a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.826 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.789+0000 7fd02e6cc640 1 --2- 172.21.15.87:0/3334752054 >> [v2:172.21.15.87:6824/2430409749,v1:172.21.15.87:6825/2430409749] conn(0x7fd00c080e50 0x7fd00c0832a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.789+0000 7fd02e6cc640 1 -- 172.21.15.87:0/3334752054 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd00c078c60 msgr2=0x7fd00c07b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.789+0000 7fd02e6cc640 1 --2- 172.21.15.87:0/3334752054 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd00c078c60 0x7fd00c07b170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd0200326a0 tx=0x7fd01c008040 comp rx=0 tx=0).stop 2024-09-17T03:14:23.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.789+0000 7fd02e6cc640 1 -- 172.21.15.87:0/3334752054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd020035220 msgr2=0x7fd020179480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.789+0000 7fd02e6cc640 1 --2- 172.21.15.87:0/3334752054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd020035220 0x7fd020179480 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd028066830 tx=0x7fd02806a0e0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.789+0000 7f326f7fe640 1 -- 172.21.15.87:0/3058792644 <== osd.5 v2:172.21.15.149:6826/2182999530 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f32880aec30 con 0x7f3274081990 2024-09-17T03:14:23.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.789+0000 7fd02cc69640 1 -- 172.21.15.87:0/3334752054 reap_dead start 2024-09-17T03:14:23.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.789+0000 7fd02e6cc640 1 -- 172.21.15.87:0/3334752054 shutdown_connections 2024-09-17T03:14:23.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.789+0000 7fd02e6cc640 1 -- 172.21.15.87:0/3334752054 >> 172.21.15.87:0/3334752054 conn(0x7fd02001eb10 msgr2=0x7fd0200071e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:23.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.790+0000 7fd02e6cc640 1 -- 172.21.15.87:0/3334752054 shutdown_connections 2024-09-17T03:14:23.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.790+0000 7fd02e6cc640 1 -- 172.21.15.87:0/3334752054 wait complete. 2024-09-17T03:14:23.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.790+0000 7f328d0b6640 1 -- 172.21.15.87:0/3058792644 >> [v2:172.21.15.149:6826/2182999530,v1:172.21.15.149:6827/2182999530] conn(0x7f3274081990 msgr2=0x7f3274083e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.790+0000 7f328d0b6640 1 --2- 172.21.15.87:0/3058792644 >> [v2:172.21.15.149:6826/2182999530,v1:172.21.15.149:6827/2182999530] conn(0x7f3274081990 0x7f3274083e80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.790+0000 7f328d0b6640 1 -- 172.21.15.87:0/3058792644 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3274078b90 msgr2=0x7f327407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.790+0000 7f328d0b6640 1 --2- 172.21.15.87:0/3058792644 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3274078b90 0x7f327407b0a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f327c000f60 tx=0x7f327c0079b0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.790+0000 7f328d0b6640 1 -- 172.21.15.87:0/3058792644 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f328815e9f0 msgr2=0x7f32880b48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.790+0000 7f328d0b6640 1 --2- 172.21.15.87:0/3058792644 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f328815e9f0 0x7f32880b48d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f327800c430 tx=0x7f327800c8f0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.790+0000 7f328759e640 1 -- 172.21.15.87:0/3058792644 reap_dead start 2024-09-17T03:14:23.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.790+0000 7f114c23f640 1 -- 172.21.15.87:0/2304947979 --> [v2:172.21.15.87:6816/4253161216,v1:172.21.15.87:6817/4253161216] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f112c002dd0 con 0x7f1128080d80 2024-09-17T03:14:23.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.790+0000 7f328d0b6640 1 -- 172.21.15.87:0/3058792644 shutdown_connections 2024-09-17T03:14:23.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.790+0000 7f328d0b6640 1 -- 172.21.15.87:0/3058792644 >> 172.21.15.87:0/3058792644 conn(0x7f32880d08c0 msgr2=0x7f3288070ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:23.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.790+0000 7f328d0b6640 1 -- 172.21.15.87:0/3058792644 shutdown_connections 2024-09-17T03:14:23.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.791+0000 7f328d0b6640 1 -- 172.21.15.87:0/3058792644 wait complete. 2024-09-17T03:14:23.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.791+0000 7f113a7fc640 1 -- 172.21.15.87:0/2304947979 <== osd.3 v2:172.21.15.87:6816/4253161216 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f112c002dd0 con 0x7f1128080d80 2024-09-17T03:14:23.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.791+0000 7f303affd640 1 -- 172.21.15.87:0/2485173823 --> [v2:172.21.15.149:6818/2771010477,v1:172.21.15.149:6819/2771010477] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f30440cc7c0 con 0x7f30440c7670 2024-09-17T03:14:23.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.791+0000 7f114c23f640 1 -- 172.21.15.87:0/2304947979 >> [v2:172.21.15.87:6816/4253161216,v1:172.21.15.87:6817/4253161216] conn(0x7f1128080d80 msgr2=0x7f11280831d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.791+0000 7f114c23f640 1 --2- 172.21.15.87:0/2304947979 >> [v2:172.21.15.87:6816/4253161216,v1:172.21.15.87:6817/4253161216] conn(0x7f1128080d80 0x7f11280831d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.792+0000 7f114c23f640 1 -- 172.21.15.87:0/2304947979 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1128078b90 msgr2=0x7f112807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.792+0000 7f30417fa640 1 -- 172.21.15.87:0/2485173823 <== osd.7 v2:172.21.15.149:6818/2771010477 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f30440cc7c0 con 0x7f30440c7670 2024-09-17T03:14:23.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.792+0000 7f114c23f640 1 --2- 172.21.15.87:0/2304947979 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1128078b90 0x7f112807b0a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1140034000 tx=0x7f1140004730 comp rx=0 tx=0).stop 2024-09-17T03:14:23.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.792+0000 7f114c23f640 1 -- 172.21.15.87:0/2304947979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f113c036970 msgr2=0x7f113c043350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.792+0000 7f114c23f640 1 --2- 172.21.15.87:0/2304947979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f113c036970 0x7f113c043350 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f113400d740 tx=0x7f113400dc00 comp rx=0 tx=0).stop 2024-09-17T03:14:23.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.792+0000 7f114a7dc640 1 -- 172.21.15.87:0/2304947979 reap_dead start 2024-09-17T03:14:23.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.792+0000 7f114c23f640 1 -- 172.21.15.87:0/2304947979 shutdown_connections 2024-09-17T03:14:23.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.792+0000 7f114c23f640 1 -- 172.21.15.87:0/2304947979 >> 172.21.15.87:0/2304947979 conn(0x7f113c051420 msgr2=0x7f113c006db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:23.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.792+0000 7fb6c9dda640 1 -- 172.21.15.87:0/2912586481 --> [v2:172.21.15.87:6800/1450834766,v1:172.21.15.87:6801/1450834766] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb6bc0541f0 con 0x7fb6a8080f60 2024-09-17T03:14:23.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.792+0000 7f114c23f640 1 -- 172.21.15.87:0/2304947979 shutdown_connections 2024-09-17T03:14:23.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.792+0000 7f114c23f640 1 -- 172.21.15.87:0/2304947979 wait complete. 2024-09-17T03:14:23.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.793+0000 7f303affd640 1 -- 172.21.15.87:0/2485173823 >> [v2:172.21.15.149:6818/2771010477,v1:172.21.15.149:6819/2771010477] conn(0x7f30440c7670 msgr2=0x7f30440c7a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.793+0000 7f303affd640 1 --2- 172.21.15.87:0/2485173823 >> [v2:172.21.15.149:6818/2771010477,v1:172.21.15.149:6819/2771010477] conn(0x7f30440c7670 0x7f30440c7a80 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.793+0000 7fb6affff640 1 -- 172.21.15.87:0/2912586481 <== osd.2 v2:172.21.15.87:6800/1450834766 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb6bc0541f0 con 0x7fb6a8080f60 2024-09-17T03:14:23.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.793+0000 7f303affd640 1 -- 172.21.15.87:0/2485173823 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3024078c90 msgr2=0x7f302407b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.793+0000 7f303affd640 1 --2- 172.21.15.87:0/2485173823 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3024078c90 0x7f302407b1a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f302c009990 tx=0x7f302c009110 comp rx=0 tx=0).stop 2024-09-17T03:14:23.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.793+0000 7f303affd640 1 -- 172.21.15.87:0/2485173823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f304415aed0 msgr2=0x7f3044167d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.793+0000 7f303affd640 1 --2- 172.21.15.87:0/2485173823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f304415aed0 0x7f3044167d70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f303c031000 tx=0x7f303c003940 comp rx=0 tx=0).stop 2024-09-17T03:14:23.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.793+0000 7f3049406640 1 -- 172.21.15.87:0/2485173823 reap_dead start 2024-09-17T03:14:23.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.793+0000 7f303affd640 1 -- 172.21.15.87:0/2485173823 shutdown_connections 2024-09-17T03:14:23.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.793+0000 7f303affd640 1 -- 172.21.15.87:0/2485173823 >> 172.21.15.87:0/2485173823 conn(0x7f304406ee70 msgr2=0x7f30440cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:23.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.794+0000 7f303affd640 1 -- 172.21.15.87:0/2485173823 shutdown_connections 2024-09-17T03:14:23.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.794+0000 7fb6adffb640 1 -- 172.21.15.87:0/2912586481 >> [v2:172.21.15.87:6800/1450834766,v1:172.21.15.87:6801/1450834766] conn(0x7fb6a8080f60 msgr2=0x7fb6a80833b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.794+0000 7fb6adffb640 1 --2- 172.21.15.87:0/2912586481 >> [v2:172.21.15.87:6800/1450834766,v1:172.21.15.87:6801/1450834766] conn(0x7fb6a8080f60 0x7fb6a80833b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.794+0000 7f303affd640 1 -- 172.21.15.87:0/2485173823 wait complete. 2024-09-17T03:14:23.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.794+0000 7fe025e8b640 1 -- 172.21.15.87:0/3195857615 --> [v2:172.21.15.149:6810/4289519146,v1:172.21.15.149:6811/4289519146] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe0200ceb30 con 0x7fe00c080e60 2024-09-17T03:14:23.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.794+0000 7fb6adffb640 1 -- 172.21.15.87:0/2912586481 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb6a8078c90 msgr2=0x7fb6a807b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.794+0000 7fb6adffb640 1 --2- 172.21.15.87:0/2912586481 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb6a8078c90 0x7fb6a807b1a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0006fd0 tx=0x7fb6b0008040 comp rx=0 tx=0).stop 2024-09-17T03:14:23.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.794+0000 7fb6adffb640 1 -- 172.21.15.87:0/2912586481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6bc03ae90 msgr2=0x7fb6bc0d7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.794+0000 7fb6adffb640 1 --2- 172.21.15.87:0/2912586481 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb6bc03ae90 0x7fb6bc0d7290 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb6b8000c00 tx=0x7fb6b8033ce0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.794+0000 7fb6c3fff640 1 -- 172.21.15.87:0/2912586481 reap_dead start 2024-09-17T03:14:23.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.795+0000 7fb6adffb640 1 -- 172.21.15.87:0/2912586481 shutdown_connections 2024-09-17T03:14:23.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.795+0000 7fb6adffb640 1 -- 172.21.15.87:0/2912586481 >> 172.21.15.87:0/2912586481 conn(0x7fb6bc01eae0 msgr2=0x7fb6bc0067e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:23.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.795+0000 7fb6adffb640 1 -- 172.21.15.87:0/2912586481 shutdown_connections 2024-09-17T03:14:23.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.795+0000 7fe007fff640 1 -- 172.21.15.87:0/3195857615 <== osd.6 v2:172.21.15.149:6810/4289519146 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe0200ceb30 con 0x7fe00c080e60 2024-09-17T03:14:23.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.795+0000 7fb6adffb640 1 -- 172.21.15.87:0/2912586481 wait complete. 2024-09-17T03:14:23.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.796+0000 7fe025e8b640 1 -- 172.21.15.87:0/3195857615 >> [v2:172.21.15.149:6810/4289519146,v1:172.21.15.149:6811/4289519146] conn(0x7fe00c080e60 msgr2=0x7fe00c0832b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.796+0000 7fe025e8b640 1 --2- 172.21.15.87:0/3195857615 >> [v2:172.21.15.149:6810/4289519146,v1:172.21.15.149:6811/4289519146] conn(0x7fe00c080e60 0x7fe00c0832b0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:23.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.796+0000 7fe025e8b640 1 -- 172.21.15.87:0/3195857615 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe00c078b90 msgr2=0x7fe00c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.796+0000 7fe025e8b640 1 --2- 172.21.15.87:0/3195857615 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe00c078b90 0x7fe00c07b0a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe010009590 tx=0x7fe010004730 comp rx=0 tx=0).stop 2024-09-17T03:14:23.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.796+0000 7fe025e8b640 1 -- 172.21.15.87:0/3195857615 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe0200cfca0 msgr2=0x7fe0201f93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:23.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.796+0000 7fe025e8b640 1 --2- 172.21.15.87:0/3195857615 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe0200cfca0 0x7fe0201f93a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe0200ba0e0 tx=0x7fe01400ea60 comp rx=0 tx=0).stop 2024-09-17T03:14:23.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.796+0000 7fe01ffff640 1 -- 172.21.15.87:0/3195857615 reap_dead start 2024-09-17T03:14:23.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.797+0000 7fe025e8b640 1 -- 172.21.15.87:0/3195857615 shutdown_connections 2024-09-17T03:14:23.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.797+0000 7fe025e8b640 1 -- 172.21.15.87:0/3195857615 >> 172.21.15.87:0/3195857615 conn(0x7fe0200d4440 msgr2=0x7fe0201e6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:23.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.797+0000 7fe025e8b640 1 -- 172.21.15.87:0/3195857615 shutdown_connections 2024-09-17T03:14:23.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:23.797+0000 7fe025e8b640 1 -- 172.21.15.87:0/3195857615 wait complete. 2024-09-17T03:14:23.842 INFO:teuthology.orchestra.run.smithi087.stdout:64424509449 2024-09-17T03:14:23.842 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T03:14:23.843 INFO:teuthology.orchestra.run.smithi087.stdout:64424509449 2024-09-17T03:14:23.843 INFO:teuthology.orchestra.run.smithi087.stdout:55834574855 2024-09-17T03:14:23.844 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T03:14:23.844 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T03:14:23.845 INFO:teuthology.orchestra.run.smithi087.stdout:64424509448 2024-09-17T03:14:23.845 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T03:14:23.846 INFO:teuthology.orchestra.run.smithi087.stdout:64424509448 2024-09-17T03:14:23.846 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T03:14:23.847 INFO:teuthology.orchestra.run.smithi087.stdout:64424509449 2024-09-17T03:14:23.847 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T03:14:23.848 INFO:teuthology.orchestra.run.smithi087.stdout:64424509449 2024-09-17T03:14:23.848 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T03:14:23.849 INFO:teuthology.orchestra.run.smithi087.stdout:64424509449 2024-09-17T03:14:23.849 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T03:14:24.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.099+0000 7f6e93f2b640 1 -- 172.21.15.87:0/14572015 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e8c158690 msgr2=0x7f6e8c0cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.099+0000 7f6e93f2b640 1 --2- 172.21.15.87:0/14572015 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e8c158690 0x7f6e8c0cc740 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c0b1540 tx=0x7f6e8802a4d0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.101+0000 7f6e93f2b640 1 -- 172.21.15.87:0/14572015 shutdown_connections 2024-09-17T03:14:24.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.101+0000 7f6e93f2b640 1 --2- 172.21.15.87:0/14572015 >> v2:172.21.15.149:3300/0 conn(0x7f6e8c073ac0 0x7f6e8c15b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.101+0000 7f6e93f2b640 1 --2- 172.21.15.87:0/14572015 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e8c158690 0x7f6e8c0cc740 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.101+0000 7f6e93f2b640 1 -- 172.21.15.87:0/14572015 >> 172.21.15.87:0/14572015 conn(0x7f6e8c06d4c0 msgr2=0x7f6e8c0d0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:24.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.101+0000 7f6e93f2b640 1 -- 172.21.15.87:0/14572015 shutdown_connections 2024-09-17T03:14:24.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.101+0000 7f6e93f2b640 1 -- 172.21.15.87:0/14572015 wait complete. 2024-09-17T03:14:24.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f6dfa29b640 1 -- 172.21.15.87:0/386210493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6de0004800 msgr2=0x7f6de0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f6dfa29b640 1 --2- 172.21.15.87:0/386210493 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6de0004800 0x7f6de0004c10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00c290 tx=0x7f6de0005830 comp rx=0 tx=0).stop 2024-09-17T03:14:24.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f47f94fc640 1 -- 172.21.15.87:0/4181141914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47ec0df800 msgr2=0x7f47ec007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f47f94fc640 1 --2- 172.21.15.87:0/4181141914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47ec0df800 0x7f47ec007c60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f47ec00e1d0 tx=0x7f47e80329d0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7fda11c4d640 1 -- 172.21.15.87:0/1076720252 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda0c0afac0 msgr2=0x7fda0c15af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7fda11c4d640 1 --2- 172.21.15.87:0/1076720252 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda0c0afac0 0x7fda0c15af00 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc009860 tx=0x7fd9fc02a490 comp rx=0 tx=0).stop 2024-09-17T03:14:24.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f6e93f2b640 1 Processor -- start 2024-09-17T03:14:24.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7fe419720640 1 -- 172.21.15.87:0/567650532 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe4140b4ee0 msgr2=0x7fe41406cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7fe419720640 1 --2- 172.21.15.87:0/567650532 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe4140b4ee0 0x7fe41406cde0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe40c005fd0 tx=0x7fe40c02a8c0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f47f94fc640 1 -- 172.21.15.87:0/4181141914 shutdown_connections 2024-09-17T03:14:24.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f47f94fc640 1 --2- 172.21.15.87:0/4181141914 >> v2:172.21.15.149:3300/0 conn(0x7f47ec053710 0x7f47ec053b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f47f94fc640 1 --2- 172.21.15.87:0/4181141914 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f47ec0519a0 0x7f47ec051db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f47f94fc640 1 --2- 172.21.15.87:0/4181141914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47ec0df800 0x7f47ec007c60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f47f94fc640 1 -- 172.21.15.87:0/4181141914 >> 172.21.15.87:0/4181141914 conn(0x7f47ec01e260 msgr2=0x7f47ec039e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:24.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f6df8838640 1 -- 172.21.15.87:0/386210493 reap_dead start 2024-09-17T03:14:24.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7fda11c4d640 1 -- 172.21.15.87:0/1076720252 shutdown_connections 2024-09-17T03:14:24.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7fda11c4d640 1 --2- 172.21.15.87:0/1076720252 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda0c0afac0 0x7fda0c15af00 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7fda11c4d640 1 -- 172.21.15.87:0/1076720252 >> 172.21.15.87:0/1076720252 conn(0x7fda0c0d08c0 msgr2=0x7fda0c0d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:24.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f47f94fc640 1 -- 172.21.15.87:0/4181141914 shutdown_connections 2024-09-17T03:14:24.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7fe419720640 1 -- 172.21.15.87:0/567650532 shutdown_connections 2024-09-17T03:14:24.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7fe419720640 1 --2- 172.21.15.87:0/567650532 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe4140b4ee0 0x7fe41406cde0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7fe419720640 1 --2- 172.21.15.87:0/567650532 >> v2:172.21.15.149:3300/0 conn(0x7fe414074f40 0x7fe4141557b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7fe419720640 1 -- 172.21.15.87:0/567650532 >> 172.21.15.87:0/567650532 conn(0x7fe4140b2d10 msgr2=0x7fe4140b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:24.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f6dfa29b640 1 -- 172.21.15.87:0/386210493 shutdown_connections 2024-09-17T03:14:24.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f6dfa29b640 1 -- 172.21.15.87:0/386210493 >> 172.21.15.87:0/386210493 conn(0x7f6df40b1fd0 msgr2=0x7f6df40cfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:24.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f1ae9a5c640 1 -- 172.21.15.87:0/3304265521 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1adc036b40 msgr2=0x7f1adc006660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f1ae9a5c640 1 --2- 172.21.15.87:0/3304265521 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1adc036b40 0x7f1adc006660 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4000b60 tx=0x7f1ae40a6430 comp rx=0 tx=0).stop 2024-09-17T03:14:24.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f05e695e640 1 -- 172.21.15.87:0/3825649953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05d8055280 msgr2=0x7f05d8007630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f05e695e640 1 --2- 172.21.15.87:0/3825649953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05d8055280 0x7f05d8007630 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f05cc009830 tx=0x7f05cc02a5b0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f6dfa29b640 1 -- 172.21.15.87:0/386210493 shutdown_connections 2024-09-17T03:14:24.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f47f94fc640 1 -- 172.21.15.87:0/4181141914 wait complete. 2024-09-17T03:14:24.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f6e93f2b640 1 -- start start 2024-09-17T03:14:24.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f1ae9a5c640 1 -- 172.21.15.87:0/3304265521 shutdown_connections 2024-09-17T03:14:24.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f1ae9a5c640 1 --2- 172.21.15.87:0/3304265521 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1adc036b40 0x7f1adc006660 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f1ae9a5c640 1 --2- 172.21.15.87:0/3304265521 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1adc009ba0 0x7f1adc0df180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f1ae9a5c640 1 -- 172.21.15.87:0/3304265521 >> 172.21.15.87:0/3304265521 conn(0x7f1adc051420 msgr2=0x7f1adc051860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:24.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f6dfa29b640 1 -- 172.21.15.87:0/386210493 wait complete. 2024-09-17T03:14:24.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f1ae9a5c640 1 -- 172.21.15.87:0/3304265521 shutdown_connections 2024-09-17T03:14:24.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f05e695e640 1 -- 172.21.15.87:0/3825649953 shutdown_connections 2024-09-17T03:14:24.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f05e695e640 1 --2- 172.21.15.87:0/3825649953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05d8055280 0x7f05d8007630 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f05e695e640 1 -- 172.21.15.87:0/3825649953 >> 172.21.15.87:0/3825649953 conn(0x7f05d801ddf0 msgr2=0x7f05d8039ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:24.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.105+0000 7f05e695e640 1 -- 172.21.15.87:0/3825649953 shutdown_connections 2024-09-17T03:14:24.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7fe419720640 1 -- 172.21.15.87:0/567650532 shutdown_connections 2024-09-17T03:14:24.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7fe419720640 1 -- 172.21.15.87:0/567650532 wait complete. 2024-09-17T03:14:24.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.105+0000 7fda11c4d640 1 -- 172.21.15.87:0/1076720252 shutdown_connections 2024-09-17T03:14:24.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.106+0000 7f98c359e640 1 -- 172.21.15.87:0/727221157 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98bc0df640 msgr2=0x7f98bc01c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.106+0000 7f98c359e640 1 --2- 172.21.15.87:0/727221157 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98bc0df640 0x7f98bc01c5b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f98c406cc30 tx=0x7f98c40a6380 comp rx=0 tx=0).stop 2024-09-17T03:14:24.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.104+0000 7f1ae9a5c640 1 -- 172.21.15.87:0/3304265521 wait complete. 2024-09-17T03:14:24.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.105+0000 7fda11c4d640 1 -- 172.21.15.87:0/1076720252 wait complete. 2024-09-17T03:14:24.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.106+0000 7f6e93f2b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e8c073ac0 0x7f6e8c14a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.106+0000 7f98c359e640 1 -- 172.21.15.87:0/727221157 shutdown_connections 2024-09-17T03:14:24.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.106+0000 7f98c359e640 1 --2- 172.21.15.87:0/727221157 >> v2:172.21.15.149:3300/0 conn(0x7f98bc01caf0 0x7f98bc00ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.106+0000 7f98c359e640 1 --2- 172.21.15.87:0/727221157 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98bc0df640 0x7f98bc01c5b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.106+0000 7f98c359e640 1 -- 172.21.15.87:0/727221157 >> 172.21.15.87:0/727221157 conn(0x7f98bc051420 msgr2=0x7f98bc051860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:24.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.107+0000 7f98c359e640 1 -- 172.21.15.87:0/727221157 shutdown_connections 2024-09-17T03:14:24.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.107+0000 7f98c359e640 1 -- 172.21.15.87:0/727221157 wait complete. 2024-09-17T03:14:24.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.107+0000 7f6e93f2b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e8c0ccc80 0x7f6e8c14a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.107+0000 7f6dfa29b640 1 Processor -- start 2024-09-17T03:14:24.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.107+0000 7f6dfa29b640 1 -- start start 2024-09-17T03:14:24.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.107+0000 7f6e914c6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e8c0ccc80 0x7f6e8c14a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.107+0000 7fe419720640 1 Processor -- start 2024-09-17T03:14:24.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.107+0000 7f1ae9a5c640 1 Processor -- start 2024-09-17T03:14:24.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.107+0000 7f6e914c6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e8c0ccc80 0x7f6e8c14a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40290/0 (socket says 172.21.15.87:40290) 2024-09-17T03:14:24.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.107+0000 7f6e91cc7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e8c073ac0 0x7f6e8c14a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.107+0000 7f6e91cc7640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e8c073ac0 0x7f6e8c14a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.87:58014/0 (socket says 172.21.15.87:58014) 2024-09-17T03:14:24.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.107+0000 7f6e914c6640 1 -- 172.21.15.87:0/2815521243 learned_addr learned my addr 172.21.15.87:0/2815521243 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:24.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.107+0000 7f6e93f2b640 1 --2- 172.21.15.87:0/2815521243 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6e8c158690 0x7f6e8c14e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.107+0000 7f6e93f2b640 1 -- 172.21.15.87:0/2815521243 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c069090 con 0x7f6e8c0ccc80 2024-09-17T03:14:24.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.107+0000 7fe419720640 1 -- start start 2024-09-17T03:14:24.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.108+0000 7f6e93f2b640 1 -- 172.21.15.87:0/2815521243 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c068f10 con 0x7f6e8c073ac0 2024-09-17T03:14:24.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.108+0000 7f6e924c8640 1 --2- 172.21.15.87:0/2815521243 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6e8c158690 0x7f6e8c14e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.108+0000 7f6e93f2b640 1 -- 172.21.15.87:0/2815521243 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6e8c069210 con 0x7f6e8c158690 2024-09-17T03:14:24.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.108+0000 7f1ae9a5c640 1 -- start start 2024-09-17T03:14:24.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.108+0000 7f6e914c6640 1 -- 172.21.15.87:0/2815521243 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6e8c158690 msgr2=0x7f6e8c14e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.108+0000 7f6e914c6640 1 --2- 172.21.15.87:0/2815521243 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6e8c158690 0x7f6e8c14e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.108+0000 7f6e914c6640 1 -- 172.21.15.87:0/2815521243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e8c073ac0 msgr2=0x7f6e8c14a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.108+0000 7f6e914c6640 1 --2- 172.21.15.87:0/2815521243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e8c073ac0 0x7f6e8c14a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.108+0000 7f6e914c6640 1 -- 172.21.15.87:0/2815521243 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e8c149820 con 0x7f6e8c0ccc80 2024-09-17T03:14:24.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.109+0000 7f6e914c6640 1 --2- 172.21.15.87:0/2815521243 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e8c0ccc80 0x7f6e8c14a840 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400caa0 tx=0x7f6e8400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:24.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.109+0000 7f6e924c8640 1 --2- 172.21.15.87:0/2815521243 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6e8c158690 0x7f6e8c14e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:14:24.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.109+0000 7f6e91cc7640 1 --2- 172.21.15.87:0/2815521243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e8c073ac0 0x7f6e8c14a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:14:24.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.109+0000 7fda11c4d640 1 Processor -- start 2024-09-17T03:14:24.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.109+0000 7fda11c4d640 1 -- start start 2024-09-17T03:14:24.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.109+0000 7fe419720640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe414074f40 0x7fe4140adc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.109+0000 7fe419720640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe4140b4ee0 0x7fe414148c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.109+0000 7fe419720640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4140b52f0 0x7fe4140af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.109+0000 7fe419720640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe41406d6b0 con 0x7fe414074f40 2024-09-17T03:14:24.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.109+0000 7fe419720640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe41406d530 con 0x7fe4140b52f0 2024-09-17T03:14:24.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.109+0000 7fe419720640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe414162c20 con 0x7fe4140b4ee0 2024-09-17T03:14:24.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.109+0000 7fe412ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe414074f40 0x7fe4140adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7fe412ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe414074f40 0x7fe4140adc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40316/0 (socket says 172.21.15.87:40316) 2024-09-17T03:14:24.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7fe412ffd640 1 -- 172.21.15.87:0/2846691704 learned_addr learned my addr 172.21.15.87:0/2846691704 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:24.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f6dfa29b640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6de0004800 0x7f6df41fdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7fe4137fe640 1 --2- 172.21.15.87:0/2846691704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4140b52f0 0x7fe4140af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f6dfa29b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6df4155130 0x7f6df4152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f6df3fff640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6de0004800 0x7f6df41fdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7fe412ffd640 1 -- 172.21.15.87:0/2846691704 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe4140b4ee0 msgr2=0x7fe414148c30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T03:14:24.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7fe412ffd640 1 --2- 172.21.15.87:0/2846691704 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe4140b4ee0 0x7fe414148c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7fe412ffd640 1 -- 172.21.15.87:0/2846691704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4140b52f0 msgr2=0x7fe4140af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7fe412ffd640 1 --2- 172.21.15.87:0/2846691704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4140b52f0 0x7fe4140af4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7fe412ffd640 1 -- 172.21.15.87:0/2846691704 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4140ae420 con 0x7fe414074f40 2024-09-17T03:14:24.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.107+0000 7f47f94fc640 1 Processor -- start 2024-09-17T03:14:24.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.107+0000 7f47f94fc640 1 -- start start 2024-09-17T03:14:24.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.105+0000 7f05e695e640 1 -- 172.21.15.87:0/3825649953 wait complete. 2024-09-17T03:14:24.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.109+0000 7f47f94fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47ec0519a0 0x7f47ec0d64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f47f3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47ec0519a0 0x7f47ec0d64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f47f3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47ec0519a0 0x7f47ec0d64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40304/0 (socket says 172.21.15.87:40304) 2024-09-17T03:14:24.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f47f3fff640 1 -- 172.21.15.87:0/3778431692 learned_addr learned my addr 172.21.15.87:0/3778431692 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:24.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f47f94fc640 1 --2- 172.21.15.87:0/3778431692 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f47ec053710 0x7f47ec04c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f47f94fc640 1 --2- 172.21.15.87:0/3778431692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47ec0df800 0x7f47ec04c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f47f94fc640 1 -- 172.21.15.87:0/3778431692 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec0541e0 con 0x7f47ec0519a0 2024-09-17T03:14:24.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f47f94fc640 1 -- 172.21.15.87:0/3778431692 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec054060 con 0x7f47ec0df800 2024-09-17T03:14:24.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f47f94fc640 1 -- 172.21.15.87:0/3778431692 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f47ec054360 con 0x7f47ec053710 2024-09-17T03:14:24.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.107+0000 7f05e695e640 1 Processor -- start 2024-09-17T03:14:24.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.108+0000 7f05e695e640 1 -- start start 2024-09-17T03:14:24.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f05e695e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05d80df390 0x7f05d8197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f05e695e640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f05d80e2d80 0x7f05d8197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f05e695e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05d80c8110 0x7f05d819b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f05e695e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f05d803a340 con 0x7f05d80df390 2024-09-17T03:14:24.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f05e695e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f05d803a1c0 con 0x7f05d80c8110 2024-09-17T03:14:24.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f05e695e640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f05d8004b40 con 0x7f05d80e2d80 2024-09-17T03:14:24.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f6df3fff640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6de0004800 0x7f6df41fdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.87:50700/0 (socket says 172.21.15.87:50700) 2024-09-17T03:14:24.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7fe412ffd640 1 --2- 172.21.15.87:0/2846691704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe414074f40 0x7fe4140adc60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe40400d740 tx=0x7fe40400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:24.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f47f3fff640 1 -- 172.21.15.87:0/3778431692 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f47ec053710 msgr2=0x7f47ec04c000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T03:14:24.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f47f3fff640 1 --2- 172.21.15.87:0/3778431692 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f47ec053710 0x7f47ec04c000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f47f3fff640 1 -- 172.21.15.87:0/3778431692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47ec0df800 msgr2=0x7f47ec04c540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-17T03:14:24.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f47f3fff640 1 --2- 172.21.15.87:0/3778431692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47ec0df800 0x7f47ec04c540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f6df3fff640 1 -- 172.21.15.87:0/3198092433 learned_addr learned my addr 172.21.15.87:0/3198092433 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:24.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f47f3fff640 1 -- 172.21.15.87:0/3778431692 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47ec04de80 con 0x7f47ec0519a0 2024-09-17T03:14:24.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f6dfa29b640 1 --2- 172.21.15.87:0/3198092433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6df41531e0 0x7f6df4158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f6deb7fe640 1 --2- 172.21.15.87:0/3198092433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6df4155130 0x7f6df4152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f6dfa29b640 1 -- 172.21.15.87:0/3198092433 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6df415a9b0 con 0x7f6df4155130 2024-09-17T03:14:24.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f6dfa29b640 1 -- 172.21.15.87:0/3198092433 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6df415a830 con 0x7f6df41531e0 2024-09-17T03:14:24.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f6dfa29b640 1 -- 172.21.15.87:0/3198092433 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6df415ab30 con 0x7f6de0004800 2024-09-17T03:14:24.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f6df8838640 1 --2- 172.21.15.87:0/3198092433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6df41531e0 0x7f6df4158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f6e827fc640 1 -- 172.21.15.87:0/2815521243 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e84003b70 con 0x7f6e8c0ccc80 2024-09-17T03:14:24.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f1ae9a5c640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1adc005c60 0x7f1adc186310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f1ae9a5c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1adc009ba0 0x7f1adc18a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f6e93f2b640 1 -- 172.21.15.87:0/2815521243 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e8c149b00 con 0x7f6e8c0ccc80 2024-09-17T03:14:24.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f1ae9a5c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1adc020c30 0x7f1adc18a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f47f3fff640 1 --2- 172.21.15.87:0/3778431692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47ec0519a0 0x7f47ec0d64e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f47e803a040 tx=0x7f47e8037740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:24.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f1ae9a5c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1adc006ba0 con 0x7f1adc009ba0 2024-09-17T03:14:24.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f1ae9a5c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1adc00be40 con 0x7f1adc020c30 2024-09-17T03:14:24.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f1ae9a5c640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1adc006d20 con 0x7f1adc005c60 2024-09-17T03:14:24.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f6e93f2b640 1 -- 172.21.15.87:0/2815521243 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e8c14cfb0 con 0x7f6e8c0ccc80 2024-09-17T03:14:24.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f98c359e640 1 Processor -- start 2024-09-17T03:14:24.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.110+0000 7f98c359e640 1 -- start start 2024-09-17T03:14:24.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f05e4efb640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05d80c8110 0x7f05d819b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f05dffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05d80df390 0x7f05d8197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f6df8838640 1 -- 172.21.15.87:0/3198092433 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6de0004800 msgr2=0x7f6df41fdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f6df8838640 1 --2- 172.21.15.87:0/3198092433 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6de0004800 0x7f6df41fdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f6df8838640 1 -- 172.21.15.87:0/3198092433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6df4155130 msgr2=0x7f6df4152a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f6df8838640 1 --2- 172.21.15.87:0/3198092433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6df4155130 0x7f6df4152a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f6df8838640 1 -- 172.21.15.87:0/3198092433 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df41fe2d0 con 0x7f6df41531e0 2024-09-17T03:14:24.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f6df3fff640 1 --2- 172.21.15.87:0/3198092433 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6de0004800 0x7f6df41fdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:14:24.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f05df7fe640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f05d80e2d80 0x7f05d8197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f05e4efb640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05d80c8110 0x7f05d819b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.87:58024/0 (socket says 172.21.15.87:58024) 2024-09-17T03:14:24.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f05e4efb640 1 -- 172.21.15.87:0/65112097 learned_addr learned my addr 172.21.15.87:0/65112097 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:24.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.112+0000 7f47f0ff9640 1 -- 172.21.15.87:0/3778431692 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e8045070 con 0x7f47ec0519a0 2024-09-17T03:14:24.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f1ae2ffd640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1adc005c60 0x7f1adc186310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.112+0000 7fda11c4d640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda0c068d00 0x7fda0c145af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7f1ae37fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1adc020c30 0x7f1adc18a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.112+0000 7f1ae37fe640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1adc020c30 0x7f1adc18a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.87:58042/0 (socket says 172.21.15.87:58042) 2024-09-17T03:14:24.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.112+0000 7f1ae37fe640 1 -- 172.21.15.87:0/2270085874 learned_addr learned my addr 172.21.15.87:0/2270085874 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:24.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.112+0000 7fda11c4d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda0c0afac0 0x7fda0c0c6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.112+0000 7fda11c4d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda0c0cc130 0x7fda0c159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.112+0000 7f47f94fc640 1 -- 172.21.15.87:0/3778431692 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47ec04e160 con 0x7f47ec0519a0 2024-09-17T03:14:24.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.112+0000 7f47f94fc640 1 -- 172.21.15.87:0/3778431692 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47ec0d6a20 con 0x7f47ec0519a0 2024-09-17T03:14:24.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.112+0000 7f47f0ff9640 1 -- 172.21.15.87:0/3778431692 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47e8037c20 con 0x7f47ec0519a0 2024-09-17T03:14:24.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.112+0000 7f47f0ff9640 1 -- 172.21.15.87:0/3778431692 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e804a6c0 con 0x7f47ec0519a0 2024-09-17T03:14:24.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.112+0000 7fda0b7fe640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda0c068d00 0x7fda0c145af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.112+0000 7fda0b7fe640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda0c068d00 0x7fda0c145af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.87:50726/0 (socket says 172.21.15.87:50726) 2024-09-17T03:14:24.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.112+0000 7fda0b7fe640 1 -- 172.21.15.87:0/270431483 learned_addr learned my addr 172.21.15.87:0/270431483 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:24.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7fe3fb7fe640 1 -- 172.21.15.87:0/2846691704 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe404017070 con 0x7fe414074f40 2024-09-17T03:14:24.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7fe3fb7fe640 1 -- 172.21.15.87:0/2846691704 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe404009d20 con 0x7fe414074f40 2024-09-17T03:14:24.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.111+0000 7fe3fb7fe640 1 -- 172.21.15.87:0/2846691704 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4040106c0 con 0x7fe414074f40 2024-09-17T03:14:24.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.112+0000 7fe419720640 1 -- 172.21.15.87:0/2846691704 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4140ae700 con 0x7fe414074f40 2024-09-17T03:14:24.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.112+0000 7fda11c4d640 1 -- 172.21.15.87:0/270431483 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c0cea30 con 0x7fda0c0afac0 2024-09-17T03:14:24.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.112+0000 7fe419720640 1 -- 172.21.15.87:0/2846691704 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4140aecc0 con 0x7fe414074f40 2024-09-17T03:14:24.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.112+0000 7fda11c4d640 1 -- 172.21.15.87:0/270431483 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fda0c069320 con 0x7fda0c0cc130 2024-09-17T03:14:24.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.113+0000 7f98c359e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98bc01caf0 0x7f98bc04fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.113+0000 7f6df8838640 1 --2- 172.21.15.87:0/3198092433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6df41531e0 0x7f6df4158330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6df40cfe40 tx=0x7f6dec00db40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:24.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.112+0000 7f1ae37fe640 1 -- 172.21.15.87:0/2270085874 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1adc005c60 msgr2=0x7f1adc186310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.113+0000 7fda11c4d640 1 -- 172.21.15.87:0/270431483 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fda0c0cebb0 con 0x7fda0c068d00 2024-09-17T03:14:24.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.113+0000 7fda0bfff640 1 --2- 172.21.15.87:0/270431483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda0c0cc130 0x7fda0c159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.113+0000 7f6e93f2b640 1 -- 172.21.15.87:0/2815521243 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e8c148840 con 0x7f6e8c0ccc80 2024-09-17T03:14:24.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.113+0000 7f98c359e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98bc038820 0x7f98bc16abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.112+0000 7f1ae37fe640 1 --2- 172.21.15.87:0/2270085874 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1adc005c60 0x7f1adc186310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.113+0000 7f6e827fc640 1 -- 172.21.15.87:0/2815521243 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e84003d10 con 0x7f6e8c0ccc80 2024-09-17T03:14:24.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.113+0000 7f05e4efb640 1 -- 172.21.15.87:0/65112097 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f05d80e2d80 msgr2=0x7f05d8197d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.113+0000 7f05e4efb640 1 --2- 172.21.15.87:0/65112097 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f05d80e2d80 0x7f05d8197d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.113+0000 7f6e827fc640 1 -- 172.21.15.87:0/2815521243 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e84005310 con 0x7f6e8c0ccc80 2024-09-17T03:14:24.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.113+0000 7f05e4efb640 1 -- 172.21.15.87:0/65112097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05d80df390 msgr2=0x7f05d8197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.113+0000 7f05e4efb640 1 --2- 172.21.15.87:0/65112097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05d80df390 0x7f05d8197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.113+0000 7f05e4efb640 1 -- 172.21.15.87:0/65112097 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d80317f0 con 0x7f05d80c8110 2024-09-17T03:14:24.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.113+0000 7f98c359e640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98bc0df640 0x7f98bc16b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.113+0000 7f6df17fa640 1 -- 172.21.15.87:0/3198092433 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec00ba10 con 0x7f6df41531e0 2024-09-17T03:14:24.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.112+0000 7f1ae37fe640 1 -- 172.21.15.87:0/2270085874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1adc009ba0 msgr2=0x7f1adc18a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.113+0000 7f98c359e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc007000 con 0x7f98bc038820 2024-09-17T03:14:24.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.113+0000 7f98c259c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98bc01caf0 0x7f98bc04fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.113+0000 7f98c359e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc006e80 con 0x7f98bc01caf0 2024-09-17T03:14:24.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.113+0000 7f98c259c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98bc01caf0 0x7f98bc04fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.87:58064/0 (socket says 172.21.15.87:58064) 2024-09-17T03:14:24.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.113+0000 7f98c259c640 1 -- 172.21.15.87:0/2163946799 learned_addr learned my addr 172.21.15.87:0/2163946799 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:24.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.113+0000 7f6dfa29b640 1 -- 172.21.15.87:0/3198092433 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6df4152fb0 con 0x7f6df41531e0 2024-09-17T03:14:24.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.113+0000 7f98c359e640 1 -- 172.21.15.87:0/2163946799 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f98bc007180 con 0x7f98bc0df640 2024-09-17T03:14:24.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.113+0000 7f98c2d9d640 1 --2- 172.21.15.87:0/2163946799 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98bc0df640 0x7f98bc16b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.112+0000 7f1ae37fe640 1 --2- 172.21.15.87:0/2270085874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1adc009ba0 0x7f1adc18a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.114+0000 7f6dfa29b640 1 -- 172.21.15.87:0/3198092433 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6df4155b80 con 0x7f6df41531e0 2024-09-17T03:14:24.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.114+0000 7f05e4efb640 1 --2- 172.21.15.87:0/65112097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05d80c8110 0x7f05d819b220 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f05e006c7a0 tx=0x7f05e006b2b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:24.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.112+0000 7f1ae37fe640 1 -- 172.21.15.87:0/2270085874 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1adc01f710 con 0x7f1adc020c30 2024-09-17T03:14:24.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.114+0000 7f1ae37fe640 1 --2- 172.21.15.87:0/2270085874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1adc020c30 0x7f1adc18a600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1ae407ac00 tx=0x7f1ae40adb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:24.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.114+0000 7f6df17fa640 1 -- 172.21.15.87:0/3198092433 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6dec019070 con 0x7f6df41531e0 2024-09-17T03:14:24.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.114+0000 7f6df17fa640 1 -- 172.21.15.87:0/3198092433 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dec010460 con 0x7f6df41531e0 2024-09-17T03:14:24.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.114+0000 7f47f94fc640 1 -- 172.21.15.87:0/3778431692 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47d4005290 con 0x7f47ec0519a0 2024-09-17T03:14:24.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.114+0000 7f98c2d9d640 1 -- 172.21.15.87:0/2163946799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98bc01caf0 msgr2=0x7f98bc04fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.114+0000 7f98c1d9b640 1 --2- 172.21.15.87:0/2163946799 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98bc038820 0x7f98bc16abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.114+0000 7f98c2d9d640 1 --2- 172.21.15.87:0/2163946799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98bc01caf0 0x7f98bc04fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.114+0000 7f98c2d9d640 1 -- 172.21.15.87:0/2163946799 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98bc038820 msgr2=0x7f98bc16abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.114+0000 7f98c2d9d640 1 --2- 172.21.15.87:0/2163946799 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98bc038820 0x7f98bc16abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.114+0000 7f98c2d9d640 1 -- 172.21.15.87:0/2163946799 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98bc0222d0 con 0x7f98bc0df640 2024-09-17T03:14:24.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.114+0000 7f98c1d9b640 1 --2- 172.21.15.87:0/2163946799 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98bc038820 0x7f98bc16abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:14:24.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.114+0000 7fe419720640 1 -- 172.21.15.87:0/2846691704 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4141630b0 con 0x7fe414074f40 2024-09-17T03:14:24.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.114+0000 7fda0affd640 1 --2- 172.21.15.87:0/270431483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda0c0afac0 0x7fda0c0c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.115+0000 7f6e827fc640 1 -- 172.21.15.87:0/2815521243 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6e84005570 con 0x7f6e8c0ccc80 2024-09-17T03:14:24.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.114+0000 7f1acbfff640 1 -- 172.21.15.87:0/2270085874 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae40bb070 con 0x7f1adc020c30 2024-09-17T03:14:24.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.115+0000 7f1ae9a5c640 1 -- 172.21.15.87:0/2270085874 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1adc01f150 con 0x7f1adc020c30 2024-09-17T03:14:24.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.115+0000 7f1ae9a5c640 1 -- 172.21.15.87:0/2270085874 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1adc0dce60 con 0x7f1adc020c30 2024-09-17T03:14:24.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.115+0000 7f1acbfff640 1 -- 172.21.15.87:0/2270085874 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ae40c0750 con 0x7f1adc020c30 2024-09-17T03:14:24.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.114+0000 7fda0b7fe640 1 -- 172.21.15.87:0/270431483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda0c0cc130 msgr2=0x7fda0c159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.114+0000 7fda0b7fe640 1 --2- 172.21.15.87:0/270431483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda0c0cc130 0x7fda0c159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.114+0000 7fda0b7fe640 1 -- 172.21.15.87:0/270431483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda0c0afac0 msgr2=0x7fda0c0c6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.114+0000 7fda0b7fe640 1 --2- 172.21.15.87:0/270431483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda0c0afac0 0x7fda0c0c6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.114+0000 7fda0b7fe640 1 -- 172.21.15.87:0/270431483 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda0c15dbd0 con 0x7fda0c068d00 2024-09-17T03:14:24.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.115+0000 7fda0affd640 1 --2- 172.21.15.87:0/270431483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda0c0afac0 0x7fda0c0c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:14:24.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.115+0000 7fda0b7fe640 1 --2- 172.21.15.87:0/270431483 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda0c068d00 0x7fda0c145af0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc0038a0 tx=0x7fd9fc0038d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:24.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.114+0000 7f05dcff9640 1 -- 172.21.15.87:0/65112097 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e007f070 con 0x7f05d80c8110 2024-09-17T03:14:24.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.114+0000 7f05dcff9640 1 -- 172.21.15.87:0/65112097 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05e0072630 con 0x7f05d80c8110 2024-09-17T03:14:24.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.114+0000 7f05dcff9640 1 -- 172.21.15.87:0/65112097 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e0072a60 con 0x7f05d80c8110 2024-09-17T03:14:24.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.115+0000 7f47f0ff9640 1 -- 172.21.15.87:0/3778431692 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f47e803e070 con 0x7f47ec0519a0 2024-09-17T03:14:24.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.116+0000 7f47f0ff9640 1 --2- 172.21.15.87:0/3778431692 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f47e4078dd0 0x7f47e407b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.116+0000 7f47f0ff9640 1 -- 172.21.15.87:0/3778431692 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f47e80c3ef0 con 0x7f47ec0519a0 2024-09-17T03:14:24.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.116+0000 7f98c2d9d640 1 --2- 172.21.15.87:0/2163946799 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98bc0df640 0x7f98bc16b0f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f98b800d5e0 tx=0x7f98b800bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:24.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.116+0000 7fd9f3fff640 1 -- 172.21.15.87:0/270431483 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc03b410 con 0x7fda0c068d00 2024-09-17T03:14:24.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.116+0000 7fd9f3fff640 1 -- 172.21.15.87:0/270431483 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9fc03b6a0 con 0x7fda0c068d00 2024-09-17T03:14:24.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.116+0000 7f6e827fc640 1 --2- 172.21.15.87:0/2815521243 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6e70078b90 0x7f6e7007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.116+0000 7f1acbfff640 1 -- 172.21.15.87:0/2270085874 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae40c0b80 con 0x7f1adc020c30 2024-09-17T03:14:24.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.116+0000 7fda11c4d640 1 -- 172.21.15.87:0/270431483 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda0c15de70 con 0x7fda0c068d00 2024-09-17T03:14:24.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.116+0000 7f6e91cc7640 1 --2- 172.21.15.87:0/2815521243 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6e70078b90 0x7f6e7007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.116+0000 7f6e827fc640 1 -- 172.21.15.87:0/2815521243 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6e8409ad90 con 0x7f6e8c0ccc80 2024-09-17T03:14:24.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.116+0000 7f1ae9a5c640 1 -- 172.21.15.87:0/2270085874 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac4005290 con 0x7f1adc020c30 2024-09-17T03:14:24.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.116+0000 7fda11c4d640 1 -- 172.21.15.87:0/270431483 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fda0c15e430 con 0x7fda0c068d00 2024-09-17T03:14:24.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.117+0000 7f98b2ffd640 1 -- 172.21.15.87:0/2163946799 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b8017070 con 0x7f98bc0df640 2024-09-17T03:14:24.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.117+0000 7f98b2ffd640 1 -- 172.21.15.87:0/2163946799 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98b8005310 con 0x7f98bc0df640 2024-09-17T03:14:24.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.117+0000 7f98b2ffd640 1 -- 172.21.15.87:0/2163946799 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b80047a0 con 0x7f98bc0df640 2024-09-17T03:14:24.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.117+0000 7f98c359e640 1 -- 172.21.15.87:0/2163946799 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98bc1817d0 con 0x7f98bc0df640 2024-09-17T03:14:24.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.117+0000 7f98c359e640 1 -- 172.21.15.87:0/2163946799 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f98bc181d70 con 0x7f98bc0df640 2024-09-17T03:14:24.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.117+0000 7f6dfa29b640 1 -- 172.21.15.87:0/3198092433 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dd8005290 con 0x7f6df41531e0 2024-09-17T03:14:24.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.116+0000 7fd9f3fff640 1 -- 172.21.15.87:0/270431483 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc03bad0 con 0x7fda0c068d00 2024-09-17T03:14:24.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.117+0000 7f6e91cc7640 1 --2- 172.21.15.87:0/2815521243 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6e70078b90 0x7f6e7007b0a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6e88033000 tx=0x7f6e88004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:24.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.117+0000 7f6df17fa640 1 -- 172.21.15.87:0/3198092433 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6dec010600 con 0x7f6df41531e0 2024-09-17T03:14:24.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.118+0000 7f6df17fa640 1 --2- 172.21.15.87:0/3198092433 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6dd4078c90 0x7f6dd407b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.118+0000 7f6df3fff640 1 --2- 172.21.15.87:0/3198092433 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6dd4078c90 0x7f6dd407b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.118+0000 7f6df17fa640 1 -- 172.21.15.87:0/3198092433 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6dec09ad30 con 0x7f6df41531e0 2024-09-17T03:14:24.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.118+0000 7f05e695e640 1 -- 172.21.15.87:0/65112097 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d8031ad0 con 0x7f05d80c8110 2024-09-17T03:14:24.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.119+0000 7fda11c4d640 1 -- 172.21.15.87:0/270431483 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda0c15af00 con 0x7fda0c068d00 2024-09-17T03:14:24.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.119+0000 7f05e695e640 1 -- 172.21.15.87:0/65112097 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d8032090 con 0x7f05d80c8110 2024-09-17T03:14:24.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.119+0000 7f98c359e640 1 -- 172.21.15.87:0/2163946799 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98bc00ae70 con 0x7f98bc0df640 2024-09-17T03:14:24.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.118+0000 7fe3fb7fe640 1 -- 172.21.15.87:0/2846691704 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe40401a070 con 0x7fe414074f40 2024-09-17T03:14:24.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.118+0000 7fe3fb7fe640 1 --2- 172.21.15.87:0/2846691704 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe400078c60 0x7fe40007b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.119+0000 7fe3fb7fe640 1 -- 172.21.15.87:0/2846691704 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe40409afa0 con 0x7fe414074f40 2024-09-17T03:14:24.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.119+0000 7fe4127fc640 1 --2- 172.21.15.87:0/2846691704 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe400078c60 0x7fe40007b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.119+0000 7f6df3fff640 1 --2- 172.21.15.87:0/3198092433 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6dd4078c90 0x7f6dd407b1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009d80 tx=0x7f6de4009110 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:24.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.119+0000 7fd9f3fff640 1 -- 172.21.15.87:0/270431483 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd9fc03bc70 con 0x7fda0c068d00 2024-09-17T03:14:24.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.119+0000 7f47f37fe640 1 --2- 172.21.15.87:0/3778431692 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f47e4078dd0 0x7f47e407b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.119+0000 7f98b2ffd640 1 -- 172.21.15.87:0/2163946799 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f98b8004940 con 0x7f98bc0df640 2024-09-17T03:14:24.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.119+0000 7fe4127fc640 1 --2- 172.21.15.87:0/2846691704 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe400078c60 0x7fe40007b170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe40c034000 tx=0x7fe40c004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:24.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.119+0000 7fd9f3fff640 1 --2- 172.21.15.87:0/270431483 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd9f8078c90 0x7fd9f807b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.120+0000 7f47f37fe640 1 --2- 172.21.15.87:0/3778431692 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f47e4078dd0 0x7f47e407b2e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f47e0005fd0 tx=0x7f47e0005b10 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:24.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.120+0000 7fd9f3fff640 1 -- 172.21.15.87:0/270431483 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd9fc0bc460 con 0x7fda0c068d00 2024-09-17T03:14:24.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.120+0000 7f98b2ffd640 1 --2- 172.21.15.87:0/2163946799 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f98a4078be0 0x7f98a407b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.120+0000 7f98b2ffd640 1 -- 172.21.15.87:0/2163946799 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f98b809ac40 con 0x7f98bc0df640 2024-09-17T03:14:24.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.120+0000 7f98c259c640 1 --2- 172.21.15.87:0/2163946799 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f98a4078be0 0x7f98a407b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.120+0000 7f05e695e640 1 -- 172.21.15.87:0/65112097 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05d8007630 con 0x7f05d80c8110 2024-09-17T03:14:24.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.120+0000 7fda0affd640 1 --2- 172.21.15.87:0/270431483 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd9f8078c90 0x7fd9f807b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.120+0000 7f1acbfff640 1 -- 172.21.15.87:0/2270085874 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1ae4063800 con 0x7f1adc020c30 2024-09-17T03:14:24.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.120+0000 7f98c259c640 1 --2- 172.21.15.87:0/2163946799 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f98a4078be0 0x7f98a407b0f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f98b40017d0 tx=0x7f98b4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:24.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.121+0000 7fda0affd640 1 --2- 172.21.15.87:0/270431483 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd9f8078c90 0x7fd9f807b1a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0cc100 tx=0x7fda00009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:24.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.121+0000 7f1acbfff640 1 --2- 172.21.15.87:0/2270085874 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1ad0078be0 0x7f1ad007b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.121+0000 7f1acbfff640 1 -- 172.21.15.87:0/2270085874 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1ae40e6080 con 0x7f1adc020c30 2024-09-17T03:14:24.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.121+0000 7f1ae2ffd640 1 --2- 172.21.15.87:0/2270085874 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1ad0078be0 0x7f1ad007b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.121+0000 7f1ae2ffd640 1 --2- 172.21.15.87:0/2270085874 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1ad0078be0 0x7f1ad007b0f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800bb30 tx=0x7f1ad800b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:24.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.122+0000 7fe3fb7fe640 1 -- 172.21.15.87:0/2846691704 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe4040633c0 con 0x7fe414074f40 2024-09-17T03:14:24.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.123+0000 7f6df17fa640 1 -- 172.21.15.87:0/3198092433 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6dec062c20 con 0x7f6df41531e0 2024-09-17T03:14:24.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.125+0000 7f05dcff9640 1 -- 172.21.15.87:0/65112097 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f05e0083400 con 0x7f05d80c8110 2024-09-17T03:14:24.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.125+0000 7f05dcff9640 1 --2- 172.21.15.87:0/65112097 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f05c4078c10 0x7f05c407b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:24.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.125+0000 7fd9f3fff640 1 -- 172.21.15.87:0/270431483 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd9fc03f030 con 0x7fda0c068d00 2024-09-17T03:14:24.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.125+0000 7f05dffff640 1 --2- 172.21.15.87:0/65112097 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f05c4078c10 0x7f05c407b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:24.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.126+0000 7f05dffff640 1 --2- 172.21.15.87:0/65112097 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f05c4078c10 0x7f05c407b120 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f05d8022a30 tx=0x7f05d4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:24.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.129+0000 7f47f0ff9640 1 -- 172.21.15.87:0/3778431692 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f47e808c3a0 con 0x7f47ec0519a0 2024-09-17T03:14:24.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.129+0000 7f05dcff9640 1 -- 172.21.15.87:0/65112097 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f05e0085de0 con 0x7f05d80c8110 2024-09-17T03:14:24.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.129+0000 7f1acbfff640 1 -- 172.21.15.87:0/2270085874 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1ae40c5060 con 0x7f1adc020c30 2024-09-17T03:14:24.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.131+0000 7f98b2ffd640 1 -- 172.21.15.87:0/2163946799 <== mon.2 v2:172.21.15.149:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f98b8062b10 con 0x7f98bc0df640 2024-09-17T03:14:24.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.132+0000 7f6e827fc640 1 -- 172.21.15.87:0/2815521243 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6e8c0cd090 con 0x7f6e8c0ccc80 2024-09-17T03:14:24.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.134+0000 7f05dcff9640 1 -- 172.21.15.87:0/65112097 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f05e00ccfe0 con 0x7f05d80c8110 2024-09-17T03:14:24.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.545+0000 7fda11c4d640 1 -- 172.21.15.87:0/270431483 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fda0c0c7230 con 0x7fda0c068d00 2024-09-17T03:14:24.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.546+0000 7f47f94fc640 1 -- 172.21.15.87:0/3778431692 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f47d4005580 con 0x7f47ec0519a0 2024-09-17T03:14:24.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.546+0000 7fd9f3fff640 1 -- 172.21.15.87:0/270431483 <== mon.2 v2:172.21.15.149:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fd9fc084350 con 0x7fda0c068d00 2024-09-17T03:14:24.545 INFO:teuthology.orchestra.run.smithi087.stdout:64424509448 2024-09-17T03:14:24.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.547+0000 7f47f0ff9640 1 -- 172.21.15.87:0/3778431692 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f47e808bca0 con 0x7f47ec0519a0 2024-09-17T03:14:24.548 INFO:teuthology.orchestra.run.smithi087.stdout:55834574854 2024-09-17T03:14:24.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.549+0000 7fe419720640 1 -- 172.21.15.87:0/2846691704 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fe4140b5890 con 0x7fe414074f40 2024-09-17T03:14:24.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.549+0000 7f6dfa29b640 1 -- 172.21.15.87:0/3198092433 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f6dd8005580 con 0x7f6df41531e0 2024-09-17T03:14:24.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.549+0000 7fe3fb7fe640 1 -- 172.21.15.87:0/2846691704 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe404062cc0 con 0x7fe414074f40 2024-09-17T03:14:24.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.549+0000 7f6df17fa640 1 -- 172.21.15.87:0/3198092433 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6dec067650 con 0x7f6df41531e0 2024-09-17T03:14:24.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.549+0000 7fda11c4d640 1 -- 172.21.15.87:0/270431483 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd9f8078c90 msgr2=0x7fd9f807b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.549+0000 7fda11c4d640 1 --2- 172.21.15.87:0/270431483 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd9f8078c90 0x7fd9f807b1a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fda0c0cc100 tx=0x7fda00009040 comp rx=0 tx=0).stop 2024-09-17T03:14:24.550 INFO:teuthology.orchestra.run.smithi087.stdout:64424509447 2024-09-17T03:14:24.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.550+0000 7fda11c4d640 1 -- 172.21.15.87:0/270431483 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda0c068d00 msgr2=0x7fda0c145af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.550+0000 7fda11c4d640 1 --2- 172.21.15.87:0/270431483 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda0c068d00 0x7fda0c145af0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc0038a0 tx=0x7fd9fc0038d0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.550 INFO:teuthology.orchestra.run.smithi087.stdout:64424509448 2024-09-17T03:14:24.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.550+0000 7fda11c4d640 1 -- 172.21.15.87:0/270431483 shutdown_connections 2024-09-17T03:14:24.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.550+0000 7fda11c4d640 1 --2- 172.21.15.87:0/270431483 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd9f8078c90 0x7fd9f807b1a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.550+0000 7fda11c4d640 1 --2- 172.21.15.87:0/270431483 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fda0c0cc130 0x7fda0c159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.550+0000 7fda11c4d640 1 --2- 172.21.15.87:0/270431483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fda0c0afac0 0x7fda0c0c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.550+0000 7fda11c4d640 1 --2- 172.21.15.87:0/270431483 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fda0c068d00 0x7fda0c145af0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.550+0000 7fda11c4d640 1 -- 172.21.15.87:0/270431483 >> 172.21.15.87:0/270431483 conn(0x7fda0c0d08c0 msgr2=0x7fda0c0cdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:24.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.550+0000 7fda11c4d640 1 -- 172.21.15.87:0/270431483 shutdown_connections 2024-09-17T03:14:24.552 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.550+0000 7fda11c4d640 1 -- 172.21.15.87:0/270431483 wait complete. 2024-09-17T03:14:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.550+0000 7f47f94fc640 1 -- 172.21.15.87:0/3778431692 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f47e4078dd0 msgr2=0x7f47e407b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.550+0000 7f47f94fc640 1 --2- 172.21.15.87:0/3778431692 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f47e4078dd0 0x7f47e407b2e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f47e0005fd0 tx=0x7f47e0005b10 comp rx=0 tx=0).stop 2024-09-17T03:14:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.551+0000 7f47f94fc640 1 -- 172.21.15.87:0/3778431692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47ec0519a0 msgr2=0x7f47ec0d64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.551+0000 7f47f94fc640 1 --2- 172.21.15.87:0/3778431692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47ec0519a0 0x7f47ec0d64e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f47e803a040 tx=0x7f47e8037740 comp rx=0 tx=0).stop 2024-09-17T03:14:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.551+0000 7f47f94fc640 1 -- 172.21.15.87:0/3778431692 shutdown_connections 2024-09-17T03:14:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.551+0000 7f47f94fc640 1 --2- 172.21.15.87:0/3778431692 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f47e4078dd0 0x7f47e407b2e0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.551+0000 7f47f94fc640 1 --2- 172.21.15.87:0/3778431692 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47ec0df800 0x7f47ec04c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.551+0000 7f47f94fc640 1 --2- 172.21.15.87:0/3778431692 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f47ec053710 0x7f47ec04c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.551+0000 7f47f94fc640 1 --2- 172.21.15.87:0/3778431692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47ec0519a0 0x7f47ec0d64e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.551+0000 7f47f94fc640 1 -- 172.21.15.87:0/3778431692 >> 172.21.15.87:0/3778431692 conn(0x7f47ec01e260 msgr2=0x7f47ec00baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.551+0000 7f47f94fc640 1 -- 172.21.15.87:0/3778431692 shutdown_connections 2024-09-17T03:14:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.551+0000 7f47f94fc640 1 -- 172.21.15.87:0/3778431692 wait complete. 2024-09-17T03:14:24.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.553+0000 7fe419720640 1 -- 172.21.15.87:0/2846691704 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe400078c60 msgr2=0x7fe40007b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.553+0000 7fe419720640 1 --2- 172.21.15.87:0/2846691704 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe400078c60 0x7fe40007b170 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe40c034000 tx=0x7fe40c004ab0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.553+0000 7fe419720640 1 -- 172.21.15.87:0/2846691704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe414074f40 msgr2=0x7fe4140adc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.553+0000 7fe419720640 1 --2- 172.21.15.87:0/2846691704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe414074f40 0x7fe4140adc60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe40400d740 tx=0x7fe40400dc00 comp rx=0 tx=0).stop 2024-09-17T03:14:24.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.553+0000 7fe419720640 1 -- 172.21.15.87:0/2846691704 shutdown_connections 2024-09-17T03:14:24.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.553+0000 7fe419720640 1 --2- 172.21.15.87:0/2846691704 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe400078c60 0x7fe40007b170 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.553+0000 7fe419720640 1 --2- 172.21.15.87:0/2846691704 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe4140b52f0 0x7fe4140af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.553+0000 7fe419720640 1 --2- 172.21.15.87:0/2846691704 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe4140b4ee0 0x7fe414148c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.553+0000 7fe419720640 1 --2- 172.21.15.87:0/2846691704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe414074f40 0x7fe4140adc60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.553+0000 7fe419720640 1 -- 172.21.15.87:0/2846691704 >> 172.21.15.87:0/2846691704 conn(0x7fe4140b2d10 msgr2=0x7fe4140cf350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:24.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.554+0000 7fe419720640 1 -- 172.21.15.87:0/2846691704 shutdown_connections 2024-09-17T03:14:24.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.554+0000 7fe419720640 1 -- 172.21.15.87:0/2846691704 wait complete. 2024-09-17T03:14:24.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.554+0000 7f6dfa29b640 1 -- 172.21.15.87:0/3198092433 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6dd4078c90 msgr2=0x7f6dd407b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.554+0000 7f6dfa29b640 1 --2- 172.21.15.87:0/3198092433 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6dd4078c90 0x7f6dd407b1a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009d80 tx=0x7f6de4009110 comp rx=0 tx=0).stop 2024-09-17T03:14:24.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.554+0000 7f6dfa29b640 1 -- 172.21.15.87:0/3198092433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6df41531e0 msgr2=0x7f6df4158330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.554+0000 7f6dfa29b640 1 --2- 172.21.15.87:0/3198092433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6df41531e0 0x7f6df4158330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6df40cfe40 tx=0x7f6dec00db40 comp rx=0 tx=0).stop 2024-09-17T03:14:24.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.554+0000 7f1ae9a5c640 1 -- 172.21.15.87:0/2270085874 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f1ac4005850 con 0x7f1adc020c30 2024-09-17T03:14:24.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.555+0000 7f6dfa29b640 1 -- 172.21.15.87:0/3198092433 shutdown_connections 2024-09-17T03:14:24.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.555+0000 7f6dfa29b640 1 --2- 172.21.15.87:0/3198092433 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6dd4078c90 0x7f6dd407b1a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.555+0000 7f6dfa29b640 1 --2- 172.21.15.87:0/3198092433 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6df41531e0 0x7f6df4158330 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.555+0000 7f6dfa29b640 1 --2- 172.21.15.87:0/3198092433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6df4155130 0x7f6df4152a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.555+0000 7f6dfa29b640 1 --2- 172.21.15.87:0/3198092433 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6de0004800 0x7f6df41fdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.555+0000 7f6dfa29b640 1 -- 172.21.15.87:0/3198092433 >> 172.21.15.87:0/3198092433 conn(0x7f6df40b1fd0 msgr2=0x7f6df4075c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:24.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.555+0000 7f6dfa29b640 1 -- 172.21.15.87:0/3198092433 shutdown_connections 2024-09-17T03:14:24.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.555+0000 7f6dfa29b640 1 -- 172.21.15.87:0/3198092433 wait complete. 2024-09-17T03:14:24.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.555+0000 7f1acbfff640 1 -- 172.21.15.87:0/2270085874 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1ae40ffa60 con 0x7f1adc020c30 2024-09-17T03:14:24.558 INFO:teuthology.orchestra.run.smithi087.stdout:64424509448 2024-09-17T03:14:24.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.557+0000 7f6e93f2b640 1 -- 172.21.15.87:0/2815521243 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f6e8c15b1e0 con 0x7f6e8c0ccc80 2024-09-17T03:14:24.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.557+0000 7f05e695e640 1 -- 172.21.15.87:0/65112097 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f05d8055710 con 0x7f05d80c8110 2024-09-17T03:14:24.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.558+0000 7f6e827fc640 1 -- 172.21.15.87:0/2815521243 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6e84063130 con 0x7f6e8c0ccc80 2024-09-17T03:14:24.559 INFO:teuthology.orchestra.run.smithi087.stdout:64424509448 2024-09-17T03:14:24.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.558+0000 7f05dcff9640 1 -- 172.21.15.87:0/65112097 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f05e00d1a10 con 0x7f05d80c8110 2024-09-17T03:14:24.560 INFO:teuthology.orchestra.run.smithi087.stdout:64424509448 2024-09-17T03:14:24.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.560+0000 7f98c359e640 1 -- 172.21.15.87:0/2163946799 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f98bc052f10 con 0x7f98bc0df640 2024-09-17T03:14:24.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.559+0000 7f1ae9a5c640 1 -- 172.21.15.87:0/2270085874 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1ad0078be0 msgr2=0x7f1ad007b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.559+0000 7f1ae9a5c640 1 --2- 172.21.15.87:0/2270085874 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1ad0078be0 0x7f1ad007b0f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800bb30 tx=0x7f1ad800b2e0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.559+0000 7f1ae9a5c640 1 -- 172.21.15.87:0/2270085874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1adc020c30 msgr2=0x7f1adc18a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.559+0000 7f1ae9a5c640 1 --2- 172.21.15.87:0/2270085874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1adc020c30 0x7f1adc18a600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1ae407ac00 tx=0x7f1ae40adb80 comp rx=0 tx=0).stop 2024-09-17T03:14:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.559+0000 7f1ae9a5c640 1 -- 172.21.15.87:0/2270085874 shutdown_connections 2024-09-17T03:14:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.559+0000 7f1ae9a5c640 1 --2- 172.21.15.87:0/2270085874 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1ad0078be0 0x7f1ad007b0f0 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800bb30 tx=0x7f1ad800b2e0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.559+0000 7f1ae9a5c640 1 --2- 172.21.15.87:0/2270085874 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1adc020c30 0x7f1adc18a600 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.559+0000 7f1ae9a5c640 1 --2- 172.21.15.87:0/2270085874 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1adc009ba0 0x7f1adc18a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.559+0000 7f1ae9a5c640 1 --2- 172.21.15.87:0/2270085874 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1adc005c60 0x7f1adc186310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.559+0000 7f1ae9a5c640 1 -- 172.21.15.87:0/2270085874 >> 172.21.15.87:0/2270085874 conn(0x7f1adc051420 msgr2=0x7f1adc0546b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.559+0000 7f1ae9a5c640 1 -- 172.21.15.87:0/2270085874 shutdown_connections 2024-09-17T03:14:24.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.559+0000 7f1ae9a5c640 1 -- 172.21.15.87:0/2270085874 wait complete. 2024-09-17T03:14:24.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.560+0000 7f98b2ffd640 1 -- 172.21.15.87:0/2163946799 <== mon.2 v2:172.21.15.149:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f98b8067540 con 0x7f98bc0df640 2024-09-17T03:14:24.562 INFO:teuthology.orchestra.run.smithi087.stdout:64424509447 2024-09-17T03:14:24.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.562+0000 7f6e93f2b640 1 -- 172.21.15.87:0/2815521243 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6e70078b90 msgr2=0x7f6e7007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.562+0000 7f05e695e640 1 -- 172.21.15.87:0/65112097 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f05c4078c10 msgr2=0x7f05c407b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.562+0000 7f05e695e640 1 --2- 172.21.15.87:0/65112097 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f05c4078c10 0x7f05c407b120 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f05d8022a30 tx=0x7f05d4009040 comp rx=0 tx=0).stop 2024-09-17T03:14:24.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.562+0000 7f05e695e640 1 -- 172.21.15.87:0/65112097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05d80c8110 msgr2=0x7f05d819b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.562+0000 7f05e695e640 1 --2- 172.21.15.87:0/65112097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05d80c8110 0x7f05d819b220 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f05e006c7a0 tx=0x7f05e006b2b0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.562+0000 7f6e93f2b640 1 --2- 172.21.15.87:0/2815521243 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6e70078b90 0x7f6e7007b0a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6e88033000 tx=0x7f6e88004730 comp rx=0 tx=0).stop 2024-09-17T03:14:24.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.562+0000 7f6e93f2b640 1 -- 172.21.15.87:0/2815521243 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e8c0ccc80 msgr2=0x7f6e8c14a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.562+0000 7f6e93f2b640 1 --2- 172.21.15.87:0/2815521243 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e8c0ccc80 0x7f6e8c14a840 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400caa0 tx=0x7f6e8400cf60 comp rx=0 tx=0).stop 2024-09-17T03:14:24.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.563+0000 7f6e93f2b640 1 -- 172.21.15.87:0/2815521243 shutdown_connections 2024-09-17T03:14:24.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.563+0000 7f6e93f2b640 1 --2- 172.21.15.87:0/2815521243 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6e70078b90 0x7f6e7007b0a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.563+0000 7f6e93f2b640 1 --2- 172.21.15.87:0/2815521243 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6e8c158690 0x7f6e8c14e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.563+0000 7f6e93f2b640 1 --2- 172.21.15.87:0/2815521243 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e8c0ccc80 0x7f6e8c14a840 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.563+0000 7f6e93f2b640 1 --2- 172.21.15.87:0/2815521243 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e8c073ac0 0x7f6e8c14a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.563+0000 7f6e93f2b640 1 -- 172.21.15.87:0/2815521243 >> 172.21.15.87:0/2815521243 conn(0x7f6e8c06d4c0 msgr2=0x7f6e8c159de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:24.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.563+0000 7f6e93f2b640 1 -- 172.21.15.87:0/2815521243 shutdown_connections 2024-09-17T03:14:24.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.563+0000 7f6e93f2b640 1 -- 172.21.15.87:0/2815521243 wait complete. 2024-09-17T03:14:24.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.563+0000 7f05e695e640 1 -- 172.21.15.87:0/65112097 shutdown_connections 2024-09-17T03:14:24.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.563+0000 7f05e695e640 1 --2- 172.21.15.87:0/65112097 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f05c4078c10 0x7f05c407b120 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.563+0000 7f05e695e640 1 --2- 172.21.15.87:0/65112097 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f05d80c8110 0x7f05d819b220 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.563+0000 7f05e695e640 1 --2- 172.21.15.87:0/65112097 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f05d80e2d80 0x7f05d8197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.563+0000 7f05e695e640 1 --2- 172.21.15.87:0/65112097 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05d80df390 0x7f05d8197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.563+0000 7f05e695e640 1 -- 172.21.15.87:0/65112097 >> 172.21.15.87:0/65112097 conn(0x7f05d801ddf0 msgr2=0x7f05d80dd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:24.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.564+0000 7f05e695e640 1 -- 172.21.15.87:0/65112097 shutdown_connections 2024-09-17T03:14:24.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.564+0000 7f05e695e640 1 -- 172.21.15.87:0/65112097 wait complete. 2024-09-17T03:14:24.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.567+0000 7f98c359e640 1 -- 172.21.15.87:0/2163946799 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f98a4078be0 msgr2=0x7f98a407b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.567+0000 7f98c359e640 1 --2- 172.21.15.87:0/2163946799 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f98a4078be0 0x7f98a407b0f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f98b40017d0 tx=0x7f98b4009040 comp rx=0 tx=0).stop 2024-09-17T03:14:24.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.567+0000 7f98c359e640 1 -- 172.21.15.87:0/2163946799 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98bc0df640 msgr2=0x7f98bc16b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:24.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.567+0000 7f98c359e640 1 --2- 172.21.15.87:0/2163946799 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98bc0df640 0x7f98bc16b0f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f98b800d5e0 tx=0x7f98b800bf90 comp rx=0 tx=0).stop 2024-09-17T03:14:24.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.567+0000 7f98c359e640 1 -- 172.21.15.87:0/2163946799 shutdown_connections 2024-09-17T03:14:24.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.567+0000 7f98c359e640 1 --2- 172.21.15.87:0/2163946799 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f98a4078be0 0x7f98a407b0f0 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f98b40017d0 tx=0x7f98b4009040 comp rx=0 tx=0).stop 2024-09-17T03:14:24.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.567+0000 7f98c359e640 1 --2- 172.21.15.87:0/2163946799 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98bc0df640 0x7f98bc16b0f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.568+0000 7f98c359e640 1 --2- 172.21.15.87:0/2163946799 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98bc038820 0x7f98bc16abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.568+0000 7f98c359e640 1 --2- 172.21.15.87:0/2163946799 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98bc01caf0 0x7f98bc04fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:24.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.568+0000 7f98c359e640 1 -- 172.21.15.87:0/2163946799 >> 172.21.15.87:0/2163946799 conn(0x7f98bc051420 msgr2=0x7f98bc0e1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:24.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.572+0000 7f98c359e640 1 -- 172.21.15.87:0/2163946799 shutdown_connections 2024-09-17T03:14:24.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:24.573+0000 7f98c359e640 1 -- 172.21.15.87:0/2163946799 wait complete. 2024-09-17T03:14:24.574 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.6 2024-09-17T03:14:24.575 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2024-09-17T03:14:24.576 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.5 2024-09-17T03:14:24.577 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.7 2024-09-17T03:14:24.577 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.2 2024-09-17T03:14:24.584 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.3 2024-09-17T03:14:24.585 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509448 for osd.1 2024-09-17T03:14:24.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.0 2024-09-17T03:14:25.575 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-17T03:14:25.576 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-17T03:14:25.576 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-17T03:14:25.577 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-17T03:14:25.578 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-17T03:14:25.585 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-17T03:14:25.585 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-17T03:14:25.588 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-17T03:14:25.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.747+0000 7f39337fe640 1 -- 172.21.15.87:0/3326453500 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c036400 con 0x7f39340cbb00 2024-09-17T03:14:25.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.747+0000 7f393b119640 1 -- 172.21.15.87:0/3326453500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39340cbb00 msgr2=0x7f39340739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:25.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.748+0000 7fceb1c30640 1 -- 172.21.15.87:0/2573414342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcea4053c00 msgr2=0x7fcea4052360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:25.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.748+0000 7fceb1c30640 1 --2- 172.21.15.87:0/2573414342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcea4053c00 0x7fcea4052360 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcea404b810 tx=0x7fcea002b7d0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.747+0000 7f393b119640 1 --2- 172.21.15.87:0/3326453500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39340cbb00 0x7f39340739a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3934072b10 tx=0x7f392c02aa60 comp rx=0 tx=0).stop 2024-09-17T03:14:25.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.748+0000 7fceb1c30640 1 -- 172.21.15.87:0/2573414342 shutdown_connections 2024-09-17T03:14:25.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.748+0000 7f393b119640 1 -- 172.21.15.87:0/3326453500 shutdown_connections 2024-09-17T03:14:25.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.748+0000 7f393b119640 1 --2- 172.21.15.87:0/3326453500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39340cbb00 0x7f39340739a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.748+0000 7f393b119640 1 -- 172.21.15.87:0/3326453500 >> 172.21.15.87:0/3326453500 conn(0x7f393406ee70 msgr2=0x7f393406f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:25.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.748+0000 7fceb1c30640 1 --2- 172.21.15.87:0/2573414342 >> v2:172.21.15.87:3300/0 conn(0x7fcea40e35c0 0x7fcea40e39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.748+0000 7fceb1c30640 1 --2- 172.21.15.87:0/2573414342 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcea4053c00 0x7fcea4052360 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.748+0000 7fceb1c30640 1 -- 172.21.15.87:0/2573414342 >> 172.21.15.87:0/2573414342 conn(0x7fcea4051420 msgr2=0x7fcea4051860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:25.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.748+0000 7fceb1c30640 1 -- 172.21.15.87:0/2573414342 shutdown_connections 2024-09-17T03:14:25.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.748+0000 7f393b119640 1 -- 172.21.15.87:0/3326453500 shutdown_connections 2024-09-17T03:14:25.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.748+0000 7fceb1c30640 1 -- 172.21.15.87:0/2573414342 wait complete. 2024-09-17T03:14:25.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.748+0000 7f393b119640 1 -- 172.21.15.87:0/3326453500 wait complete. 2024-09-17T03:14:25.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.751+0000 7f393b119640 1 Processor -- start 2024-09-17T03:14:25.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.751+0000 7f393b119640 1 -- start start 2024-09-17T03:14:25.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.751+0000 7fceb1c30640 1 Processor -- start 2024-09-17T03:14:25.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.752+0000 7fceb1c30640 1 -- start start 2024-09-17T03:14:25.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.754+0000 7f393b119640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39340cb6f0 0x7f3934202a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.754+0000 7f393b119640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39340cbb00 0x7f393415ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.754+0000 7fceb1c30640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcea400ae50 0x7fcea417e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.755+0000 7f3933fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39340cbb00 0x7f393415ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:25.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.755+0000 7f3938eb5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39340cb6f0 0x7f3934202a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:25.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.755+0000 7f3938eb5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39340cb6f0 0x7f3934202a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.87:58810/0 (socket says 172.21.15.87:58810) 2024-09-17T03:14:25.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.755+0000 7f3933fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39340cbb00 0x7f393415ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50748/0 (socket says 172.21.15.87:50748) 2024-09-17T03:14:25.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.755+0000 7f3938eb5640 1 -- 172.21.15.87:0/3413325153 learned_addr learned my addr 172.21.15.87:0/3413325153 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:25.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.755+0000 7f393b119640 1 --2- 172.21.15.87:0/3413325153 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3934165a20 0x7f393415b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.755+0000 7fceab7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcea400ae50 0x7fcea417e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:25.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.755+0000 7f393b119640 1 -- 172.21.15.87:0/3413325153 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3934074060 con 0x7f39340cbb00 2024-09-17T03:14:25.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.755+0000 7f393b119640 1 -- 172.21.15.87:0/3413325153 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3934073ee0 con 0x7f39340cb6f0 2024-09-17T03:14:25.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.755+0000 7f393b119640 1 -- 172.21.15.87:0/3413325153 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f39340741e0 con 0x7f3934165a20 2024-09-17T03:14:25.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.755+0000 7fceb1c30640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcea4053c00 0x7fcea40dc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.755+0000 7fceb1c30640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcea40e35c0 0x7fcea40db600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.755+0000 7fceb1c30640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcea40528a0 con 0x7fcea400ae50 2024-09-17T03:14:25.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.756+0000 7fceab7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcea400ae50 0x7fcea417e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50760/0 (socket says 172.21.15.87:50760) 2024-09-17T03:14:25.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.756+0000 7fceab7fe640 1 -- 172.21.15.87:0/3686307917 learned_addr learned my addr 172.21.15.87:0/3686307917 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:25.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.756+0000 7fceb1c30640 1 -- 172.21.15.87:0/3686307917 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcea40388a0 con 0x7fcea40e35c0 2024-09-17T03:14:25.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.756+0000 7f3933fff640 1 -- 172.21.15.87:0/3413325153 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3934165a20 msgr2=0x7f393415b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:25.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.756+0000 7fceb1c30640 1 -- 172.21.15.87:0/3686307917 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcea4052a20 con 0x7fcea4053c00 2024-09-17T03:14:25.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.756+0000 7f3933fff640 1 --2- 172.21.15.87:0/3413325153 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3934165a20 0x7f393415b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.756+0000 7f3933fff640 1 -- 172.21.15.87:0/3413325153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39340cb6f0 msgr2=0x7f3934202a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:25.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.756+0000 7f3933fff640 1 --2- 172.21.15.87:0/3413325153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39340cb6f0 0x7f3934202a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.756+0000 7f3933fff640 1 -- 172.21.15.87:0/3413325153 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3934202fc0 con 0x7f39340cbb00 2024-09-17T03:14:25.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.757+0000 7fcad9377640 1 -- 172.21.15.87:0/208397869 reap_dead start 2024-09-17T03:14:25.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.756+0000 7fceab7fe640 1 -- 172.21.15.87:0/3686307917 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcea4053c00 msgr2=0x7fcea40dc190 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T03:14:25.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.756+0000 7fceab7fe640 1 --2- 172.21.15.87:0/3686307917 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcea4053c00 0x7fcea40dc190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.756+0000 7fceab7fe640 1 -- 172.21.15.87:0/3686307917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcea40e35c0 msgr2=0x7fcea40db600 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T03:14:25.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.757+0000 7fceab7fe640 1 --2- 172.21.15.87:0/3686307917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcea40e35c0 0x7fcea40db600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.757+0000 7fceab7fe640 1 -- 172.21.15.87:0/3686307917 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea40dca20 con 0x7fcea400ae50 2024-09-17T03:14:25.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.756+0000 7f3933fff640 1 --2- 172.21.15.87:0/3413325153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39340cbb00 0x7f393415ace0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f392800b780 tx=0x7f392800bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:25.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.757+0000 7f39317fa640 1 -- 172.21.15.87:0/3413325153 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3928019070 con 0x7f39340cbb00 2024-09-17T03:14:25.758 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.757+0000 7f39317fa640 1 -- 172.21.15.87:0/3413325153 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f392800c810 con 0x7f39340cbb00 2024-09-17T03:14:25.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.757+0000 7fceab7fe640 1 --2- 172.21.15.87:0/3686307917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcea400ae50 0x7fcea417e840 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcea002bdc0 tx=0x7fcea0030c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:25.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.757+0000 7f39317fa640 1 -- 172.21.15.87:0/3413325153 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392800cc40 con 0x7f39340cbb00 2024-09-17T03:14:25.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.758+0000 7fcadadda640 1 -- 172.21.15.87:0/208397869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcac4004800 msgr2=0x7fcac4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:25.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.757+0000 7f393b119640 1 -- 172.21.15.87:0/3413325153 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3934164980 con 0x7f39340cbb00 2024-09-17T03:14:25.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.757+0000 7f393b119640 1 -- 172.21.15.87:0/3413325153 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3934164f40 con 0x7f39340cbb00 2024-09-17T03:14:25.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.757+0000 7fce97fff640 1 -- 172.21.15.87:0/3686307917 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0040020 con 0x7fcea400ae50 2024-09-17T03:14:25.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.757+0000 7fce97fff640 1 -- 172.21.15.87:0/3686307917 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcea0005250 con 0x7fcea400ae50 2024-09-17T03:14:25.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.759+0000 7fcadadda640 1 --2- 172.21.15.87:0/208397869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcac4004800 0x7fcac4004c10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcac000c760 tx=0x7fcac4005830 comp rx=0 tx=0).stop 2024-09-17T03:14:25.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.759+0000 7f393b119640 1 -- 172.21.15.87:0/3413325153 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39341646f0 con 0x7f39340cbb00 2024-09-17T03:14:25.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.757+0000 7fce97fff640 1 -- 172.21.15.87:0/3686307917 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcea0005680 con 0x7fcea400ae50 2024-09-17T03:14:25.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.759+0000 7fceb1c30640 1 -- 172.21.15.87:0/3686307917 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea40dcd00 con 0x7fcea400ae50 2024-09-17T03:14:25.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.759+0000 7fceb1c30640 1 -- 172.21.15.87:0/3686307917 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea4171e90 con 0x7fcea400ae50 2024-09-17T03:14:25.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.761+0000 7fceb1c30640 1 -- 172.21.15.87:0/3686307917 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcea40075e0 con 0x7fcea400ae50 2024-09-17T03:14:25.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.763+0000 7fcadadda640 1 -- 172.21.15.87:0/208397869 shutdown_connections 2024-09-17T03:14:25.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.763+0000 7fcadadda640 1 --2- 172.21.15.87:0/208397869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcac4004800 0x7fcac4004c10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.763+0000 7fcadadda640 1 -- 172.21.15.87:0/208397869 >> 172.21.15.87:0/208397869 conn(0x7fcad40d08c0 msgr2=0x7fcad40d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:25.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.762+0000 7f39317fa640 1 -- 172.21.15.87:0/3413325153 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f392801c070 con 0x7f39340cbb00 2024-09-17T03:14:25.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.763+0000 7f39317fa640 1 --2- 172.21.15.87:0/3413325153 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3920078c90 0x7f392007b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.763+0000 7f39317fa640 1 -- 172.21.15.87:0/3413325153 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f392809a430 con 0x7f39340cbb00 2024-09-17T03:14:25.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.763+0000 7fce97fff640 1 -- 172.21.15.87:0/3686307917 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcea0043020 con 0x7fcea400ae50 2024-09-17T03:14:25.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.764+0000 7fce97fff640 1 --2- 172.21.15.87:0/3686307917 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fce90078d80 0x7fce9007b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.764+0000 7fcadadda640 1 -- 172.21.15.87:0/208397869 shutdown_connections 2024-09-17T03:14:25.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.764+0000 7fcadadda640 1 -- 172.21.15.87:0/208397869 wait complete. 2024-09-17T03:14:25.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.764+0000 7fceaaffd640 1 --2- 172.21.15.87:0/3686307917 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fce90078d80 0x7fce9007b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:25.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.765+0000 7f3938eb5640 1 --2- 172.21.15.87:0/3413325153 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3920078c90 0x7f392007b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:25.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.765+0000 7fceaaffd640 1 --2- 172.21.15.87:0/3686307917 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fce90078d80 0x7fce9007b290 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fce98000f60 tx=0x7fce98001510 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:25.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.765+0000 7f3938eb5640 1 --2- 172.21.15.87:0/3413325153 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3920078c90 0x7f392007b1a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3924000f60 tx=0x7f3924009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:25.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.767+0000 7fcadadda640 1 Processor -- start 2024-09-17T03:14:25.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.768+0000 7fcadadda640 1 -- start start 2024-09-17T03:14:25.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.770+0000 7fcadadda640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac4004800 0x7fcad41e9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.770+0000 7fcadadda640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcad414a9b0 0x7fcad414b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.770+0000 7fcadadda640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcad414adc0 0x7fcad414b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.770+0000 7fce97fff640 1 -- 172.21.15.87:0/3686307917 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fcea0080760 con 0x7fcea400ae50 2024-09-17T03:14:25.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.771+0000 7f39317fa640 1 -- 172.21.15.87:0/3413325153 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3928023020 con 0x7f39340cbb00 2024-09-17T03:14:25.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.771+0000 7fcad8b76640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac4004800 0x7fcad41e9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:25.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.771+0000 7fcadadda640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcad40af4c0 con 0x7fcad414adc0 2024-09-17T03:14:25.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.771+0000 7fcadadda640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcad40af340 con 0x7fcac4004800 2024-09-17T03:14:25.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.771+0000 7fcadadda640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcad40af640 con 0x7fcad414a9b0 2024-09-17T03:14:25.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.771+0000 7fcad8b76640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac4004800 0x7fcad41e9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.87:58816/0 (socket says 172.21.15.87:58816) 2024-09-17T03:14:25.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.771+0000 7fcad8b76640 1 -- 172.21.15.87:0/895224512 learned_addr learned my addr 172.21.15.87:0/895224512 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:25.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.771+0000 7fcad3fff640 1 --2- 172.21.15.87:0/895224512 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcad414a9b0 0x7fcad414b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:25.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.771+0000 7fcad9377640 1 --2- 172.21.15.87:0/895224512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcad414adc0 0x7fcad414b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:25.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.772+0000 7fcad9377640 1 -- 172.21.15.87:0/895224512 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcad414a9b0 msgr2=0x7fcad414b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:25.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.772+0000 7fcad9377640 1 --2- 172.21.15.87:0/895224512 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcad414a9b0 0x7fcad414b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.772+0000 7fcad9377640 1 -- 172.21.15.87:0/895224512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac4004800 msgr2=0x7fcad41e9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:25.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.772+0000 7fcad9377640 1 --2- 172.21.15.87:0/895224512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac4004800 0x7fcad41e9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.772+0000 7fcad9377640 1 -- 172.21.15.87:0/895224512 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad41e8670 con 0x7fcad414adc0 2024-09-17T03:14:25.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.772+0000 7fcad9377640 1 --2- 172.21.15.87:0/895224512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcad414adc0 0x7fcad414b860 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcad4071c10 tx=0x7fcacc007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:25.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.780+0000 7fcad17fa640 1 -- 172.21.15.87:0/895224512 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc00ebb0 con 0x7fcad414adc0 2024-09-17T03:14:25.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.780+0000 7fcad17fa640 1 -- 172.21.15.87:0/895224512 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcacc018070 con 0x7fcad414adc0 2024-09-17T03:14:25.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.780+0000 7fce97fff640 1 -- 172.21.15.87:0/3686307917 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fcea004a020 con 0x7fcea400ae50 2024-09-17T03:14:25.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.780+0000 7fcad17fa640 1 -- 172.21.15.87:0/895224512 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc0116b0 con 0x7fcad414adc0 2024-09-17T03:14:25.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.780+0000 7fcadadda640 1 -- 172.21.15.87:0/895224512 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad41e8950 con 0x7fcad414adc0 2024-09-17T03:14:25.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.780+0000 7fcadadda640 1 -- 172.21.15.87:0/895224512 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad41e8f10 con 0x7fcad414adc0 2024-09-17T03:14:25.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.783+0000 7fcadadda640 1 -- 172.21.15.87:0/895224512 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad40af9c0 con 0x7fcad414adc0 2024-09-17T03:14:25.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.795+0000 7fcad17fa640 1 -- 172.21.15.87:0/895224512 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcacc011850 con 0x7fcad414adc0 2024-09-17T03:14:25.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.796+0000 7fcad17fa640 1 --2- 172.21.15.87:0/895224512 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcab8078b90 0x7fcab807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.796+0000 7fcad17fa640 1 -- 172.21.15.87:0/895224512 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fcacc09bc60 con 0x7fcad414adc0 2024-09-17T03:14:25.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.796+0000 7fcad17fa640 1 -- 172.21.15.87:0/895224512 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fcacc09c120 con 0x7fcad414adc0 2024-09-17T03:14:25.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.797+0000 7fcad8b76640 1 --2- 172.21.15.87:0/895224512 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcab8078b90 0x7fcab807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:25.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.800+0000 7fcad8b76640 1 --2- 172.21.15.87:0/895224512 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcab8078b90 0x7fcab807b0a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcac8005fd0 tx=0x7fcac800b3c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:25.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.804+0000 7fdd01558640 1 -- 172.21.15.87:0/1642378926 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcec003a00 msgr2=0x7fdcec003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:25.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.804+0000 7fdd01558640 1 --2- 172.21.15.87:0/1642378926 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcec003a00 0x7fdcec003e70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000c310 tx=0x7fdcec005830 comp rx=0 tx=0).stop 2024-09-17T03:14:25.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.804+0000 7fdd01558640 1 -- 172.21.15.87:0/1642378926 shutdown_connections 2024-09-17T03:14:25.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.804+0000 7fdd01558640 1 --2- 172.21.15.87:0/1642378926 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcec003a00 0x7fdcec003e70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.804+0000 7fdd01558640 1 -- 172.21.15.87:0/1642378926 >> 172.21.15.87:0/1642378926 conn(0x7fdcf4051420 msgr2=0x7fdcf4051860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:25.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.804+0000 7fdd01558640 1 -- 172.21.15.87:0/1642378926 shutdown_connections 2024-09-17T03:14:25.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.804+0000 7fdd01558640 1 -- 172.21.15.87:0/1642378926 wait complete. 2024-09-17T03:14:25.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.807+0000 7fdd01558640 1 Processor -- start 2024-09-17T03:14:25.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.808+0000 7fdd01558640 1 -- start start 2024-09-17T03:14:25.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.810+0000 7fdd01558640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdcec003a00 0x7fdcf40e36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.810+0000 7fdd01558640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcf4181540 0x7fdcf4181950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.810+0000 7fdcfb7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcf4181540 0x7fdcf4181950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:25.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.810+0000 7fdcfb7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcf4181540 0x7fdcf4181950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50790/0 (socket says 172.21.15.87:50790) 2024-09-17T03:14:25.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.810+0000 7fdcfb7fe640 1 -- 172.21.15.87:0/27768478 learned_addr learned my addr 172.21.15.87:0/27768478 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:25.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.810+0000 7fdd01558640 1 --2- 172.21.15.87:0/27768478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcf4181e90 0x7fdcf416d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.811+0000 7fdd01558640 1 -- 172.21.15.87:0/27768478 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf401d490 con 0x7fdcf4181540 2024-09-17T03:14:25.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.811+0000 7fdd01558640 1 -- 172.21.15.87:0/27768478 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf401d310 con 0x7fdcf4181e90 2024-09-17T03:14:25.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.811+0000 7fdd01558640 1 -- 172.21.15.87:0/27768478 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fdcf4036210 con 0x7fdcec003a00 2024-09-17T03:14:25.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.811+0000 7fdcfb7fe640 1 -- 172.21.15.87:0/27768478 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdcec003a00 msgr2=0x7fdcf40e36f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T03:14:25.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.811+0000 7fdcfb7fe640 1 --2- 172.21.15.87:0/27768478 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdcec003a00 0x7fdcf40e36f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.811+0000 7fdcfb7fe640 1 -- 172.21.15.87:0/27768478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcf4181e90 msgr2=0x7fdcf416d5e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T03:14:25.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.811+0000 7fdcfb7fe640 1 --2- 172.21.15.87:0/27768478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcf4181e90 0x7fdcf416d5e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.811+0000 7fdcfb7fe640 1 -- 172.21.15.87:0/27768478 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf4054310 con 0x7fdcf4181540 2024-09-17T03:14:25.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.812+0000 7fdcfb7fe640 1 --2- 172.21.15.87:0/27768478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcf4181540 0x7fdcf4181950 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdce800d780 tx=0x7fdce800dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:25.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.813+0000 7fdcf8ff9640 1 -- 172.21.15.87:0/27768478 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8028070 con 0x7fdcf4181540 2024-09-17T03:14:25.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.813+0000 7fdcf8ff9640 1 -- 172.21.15.87:0/27768478 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdce800de90 con 0x7fdcf4181540 2024-09-17T03:14:25.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.813+0000 7fdd01558640 1 -- 172.21.15.87:0/27768478 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf416c820 con 0x7fdcf4181540 2024-09-17T03:14:25.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.816+0000 7fdcf8ff9640 1 -- 172.21.15.87:0/27768478 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8009db0 con 0x7fdcf4181540 2024-09-17T03:14:25.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.816+0000 7fdcf8ff9640 1 -- 172.21.15.87:0/27768478 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdce802b070 con 0x7fdcf4181540 2024-09-17T03:14:25.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.816+0000 7fdd01558640 1 -- 172.21.15.87:0/27768478 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf416cde0 con 0x7fdcf4181540 2024-09-17T03:14:25.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.817+0000 7fdcf8ff9640 1 --2- 172.21.15.87:0/27768478 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdce0078b70 0x7fdce007b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.817+0000 7fdcfbfff640 1 --2- 172.21.15.87:0/27768478 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdce0078b70 0x7fdce007b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:25.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.819+0000 7fdcfbfff640 1 --2- 172.21.15.87:0/27768478 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdce0078b70 0x7fdce007b080 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000c2e0 tx=0x7fdcf0004240 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:25.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.819+0000 7fdcf8ff9640 1 -- 172.21.15.87:0/27768478 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fdce80b0060 con 0x7fdcf4181540 2024-09-17T03:14:25.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.819+0000 7fdd01558640 1 -- 172.21.15.87:0/27768478 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcf4036700 con 0x7fdcf4181540 2024-09-17T03:14:25.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.825+0000 7fa7088de640 1 -- 172.21.15.87:0/841396580 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa704071ed0 msgr2=0x7fa7040b0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:25.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.825+0000 7fa7088de640 1 --2- 172.21.15.87:0/841396580 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa704071ed0 0x7fa7040b0710 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc005fd0 tx=0x7fa6fc02a8e0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.825+0000 7fa7088de640 1 -- 172.21.15.87:0/841396580 shutdown_connections 2024-09-17T03:14:25.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.825+0000 7fa7088de640 1 --2- 172.21.15.87:0/841396580 >> v2:172.21.15.149:3300/0 conn(0x7fa70415f890 0x7fa70415bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.825+0000 7fa7088de640 1 --2- 172.21.15.87:0/841396580 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa704071ed0 0x7fa7040b0710 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.825+0000 7fa7088de640 1 -- 172.21.15.87:0/841396580 >> 172.21.15.87:0/841396580 conn(0x7fa7040b2d10 msgr2=0x7fa7040b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:25.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.825+0000 7fa7088de640 1 -- 172.21.15.87:0/841396580 shutdown_connections 2024-09-17T03:14:25.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.825+0000 7fa7088de640 1 -- 172.21.15.87:0/841396580 wait complete. 2024-09-17T03:14:25.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.828+0000 7fa7088de640 1 Processor -- start 2024-09-17T03:14:25.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.829+0000 7fa7088de640 1 -- start start 2024-09-17T03:14:25.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.830+0000 7fdcf8ff9640 1 -- 172.21.15.87:0/27768478 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fdce8078e20 con 0x7fdcf4181540 2024-09-17T03:14:25.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.831+0000 7fa7088de640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa704155ab0 0x7fa704139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.831+0000 7fa7088de640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa704153c70 0x7fa70413c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.831+0000 7fa7088de640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa704154080 0x7fa70413ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.831+0000 7fa70259c640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa704155ab0 0x7fa704139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:25.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.832+0000 7fa701d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa704153c70 0x7fa70413c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:25.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.832+0000 7fa701d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa704153c70 0x7fa70413c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50806/0 (socket says 172.21.15.87:50806) 2024-09-17T03:14:25.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.832+0000 7fa7088de640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa70415c250 con 0x7fa704153c70 2024-09-17T03:14:25.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.832+0000 7fa7088de640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa70415c0d0 con 0x7fa704154080 2024-09-17T03:14:25.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.832+0000 7fa7088de640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa70415c3d0 con 0x7fa704155ab0 2024-09-17T03:14:25.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.832+0000 7fa702d9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa704154080 0x7fa70413ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:25.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.832+0000 7fa702d9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa704154080 0x7fa70413ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.87:58866/0 (socket says 172.21.15.87:58866) 2024-09-17T03:14:25.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.832+0000 7fa702d9d640 1 -- 172.21.15.87:0/2462460315 learned_addr learned my addr 172.21.15.87:0/2462460315 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:25.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.833+0000 7fa702d9d640 1 -- 172.21.15.87:0/2462460315 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa704155ab0 msgr2=0x7fa704139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:25.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.833+0000 7fa702d9d640 1 --2- 172.21.15.87:0/2462460315 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa704155ab0 0x7fa704139ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.833+0000 7fa702d9d640 1 -- 172.21.15.87:0/2462460315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa704153c70 msgr2=0x7fa70413c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:25.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.833+0000 7fa702d9d640 1 --2- 172.21.15.87:0/2462460315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa704153c70 0x7fa70413c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.833+0000 7fa702d9d640 1 -- 172.21.15.87:0/2462460315 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa704139ff0 con 0x7fa704154080 2024-09-17T03:14:25.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.833+0000 7fa702d9d640 1 --2- 172.21.15.87:0/2462460315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa704154080 0x7fa70413ca90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa6f800e6d0 tx=0x7fa6f800eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:25.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.834+0000 7fa6f2ffd640 1 -- 172.21.15.87:0/2462460315 <== mon.1 v2:172.21.15.149:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f800cab0 con 0x7fa704154080 2024-09-17T03:14:25.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.834+0000 7fa6f2ffd640 1 -- 172.21.15.87:0/2462460315 <== mon.1 v2:172.21.15.149:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6f800cc50 con 0x7fa704154080 2024-09-17T03:14:25.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.834+0000 7fa6f2ffd640 1 -- 172.21.15.87:0/2462460315 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f8005280 con 0x7fa704154080 2024-09-17T03:14:25.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.834+0000 7fa7088de640 1 -- 172.21.15.87:0/2462460315 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa70413a3b0 con 0x7fa704154080 2024-09-17T03:14:25.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.834+0000 7fa7088de640 1 -- 172.21.15.87:0/2462460315 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa70413a970 con 0x7fa704154080 2024-09-17T03:14:25.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.835+0000 7fa6f0ff9640 1 -- 172.21.15.87:0/2462460315 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa70415c960 con 0x7fa704154080 2024-09-17T03:14:25.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.837+0000 7fa6f2ffd640 1 -- 172.21.15.87:0/2462460315 <== mon.1 v2:172.21.15.149:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa6f8005420 con 0x7fa704154080 2024-09-17T03:14:25.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.838+0000 7fa6f2ffd640 1 --2- 172.21.15.87:0/2462460315 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa6e4078c60 0x7fa6e407b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.838+0000 7fa6f2ffd640 1 -- 172.21.15.87:0/2462460315 <== mon.1 v2:172.21.15.149:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa6f809b580 con 0x7fa704154080 2024-09-17T03:14:25.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.838+0000 7fa70259c640 1 --2- 172.21.15.87:0/2462460315 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa6e4078c60 0x7fa6e407b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:25.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.839+0000 7fa70259c640 1 --2- 172.21.15.87:0/2462460315 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa6e4078c60 0x7fa6e407b170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc034000 tx=0x7fa6fc033040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:25.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.843+0000 7fa6f2ffd640 1 -- 172.21.15.87:0/2462460315 <== mon.1 v2:172.21.15.149:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa6f800ce20 con 0x7fa704154080 2024-09-17T03:14:25.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.862+0000 7f3be5e97640 1 -- 172.21.15.87:0/3431984255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3be01581b0 msgr2=0x7f3be006e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:25.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.862+0000 7f3be5e97640 1 --2- 172.21.15.87:0/3431984255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3be01581b0 0x7f3be006e820 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009830 tx=0x7f3bd802a3f0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.862+0000 7fe16eb19640 1 -- 172.21.15.87:0/2667516474 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1681558c0 msgr2=0x7fe1680b4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:25.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.862+0000 7f3be5e97640 1 -- 172.21.15.87:0/3431984255 shutdown_connections 2024-09-17T03:14:25.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.862+0000 7f3be5e97640 1 --2- 172.21.15.87:0/3431984255 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3be01581b0 0x7f3be006e820 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.862+0000 7f3be5e97640 1 -- 172.21.15.87:0/3431984255 >> 172.21.15.87:0/3431984255 conn(0x7f3be00b2290 msgr2=0x7f3be00cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:25.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.862+0000 7fe16eb19640 1 --2- 172.21.15.87:0/2667516474 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1681558c0 0x7fe1680b4a90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe168069950 tx=0x7fe16002a810 comp rx=0 tx=0).stop 2024-09-17T03:14:25.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.862+0000 7fe16eb19640 1 -- 172.21.15.87:0/2667516474 shutdown_connections 2024-09-17T03:14:25.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.862+0000 7fe16eb19640 1 --2- 172.21.15.87:0/2667516474 >> v2:172.21.15.87:3300/0 conn(0x7fe16806d320 0x7fe168155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.862+0000 7f16a5ffb640 1 -- 172.21.15.87:0/3914456578 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c031850 con 0x7f16a80c5ec0 2024-09-17T03:14:25.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.863+0000 7f16ad2b6640 1 -- 172.21.15.87:0/3914456578 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a80c5ec0 msgr2=0x7f16a81684b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:25.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.863+0000 7f16ad2b6640 1 --2- 172.21.15.87:0/3914456578 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a80c5ec0 0x7f16a81684b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f16a80d0e80 tx=0x7f169c02a4d0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.862+0000 7fe16eb19640 1 --2- 172.21.15.87:0/2667516474 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1681558c0 0x7fe1680b4a90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.862+0000 7fe16eb19640 1 --2- 172.21.15.87:0/2667516474 >> v2:172.21.15.149:3300/0 conn(0x7fe168074e40 0x7fe16806cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.862+0000 7fe16eb19640 1 -- 172.21.15.87:0/2667516474 >> 172.21.15.87:0/2667516474 conn(0x7fe1680b2d10 msgr2=0x7fe1680b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:25.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.862+0000 7fe16eb19640 1 -- 172.21.15.87:0/2667516474 shutdown_connections 2024-09-17T03:14:25.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.862+0000 7fe16eb19640 1 -- 172.21.15.87:0/2667516474 wait complete. 2024-09-17T03:14:25.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.862+0000 7f3be5e97640 1 -- 172.21.15.87:0/3431984255 shutdown_connections 2024-09-17T03:14:25.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.864+0000 7f16ad2b6640 1 -- 172.21.15.87:0/3914456578 shutdown_connections 2024-09-17T03:14:25.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.864+0000 7f16ad2b6640 1 --2- 172.21.15.87:0/3914456578 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a80c5ec0 0x7f16a81684b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.864+0000 7f16ad2b6640 1 --2- 172.21.15.87:0/3914456578 >> v2:172.21.15.149:3300/0 conn(0x7f16a80aee70 0x7f16a80c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.864+0000 7f16ad2b6640 1 -- 172.21.15.87:0/3914456578 >> 172.21.15.87:0/3914456578 conn(0x7f16a80d0040 msgr2=0x7f16a80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:25.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.862+0000 7f3be5e97640 1 -- 172.21.15.87:0/3431984255 wait complete. 2024-09-17T03:14:25.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.864+0000 7f16ad2b6640 1 -- 172.21.15.87:0/3914456578 shutdown_connections 2024-09-17T03:14:25.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.864+0000 7f16ad2b6640 1 -- 172.21.15.87:0/3914456578 wait complete. 2024-09-17T03:14:25.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.865+0000 7f3be5e97640 1 Processor -- start 2024-09-17T03:14:25.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.865+0000 7f3be5e97640 1 -- start start 2024-09-17T03:14:25.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.868+0000 7f16ad2b6640 1 Processor -- start 2024-09-17T03:14:25.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.868+0000 7f16ad2b6640 1 -- start start 2024-09-17T03:14:25.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.868+0000 7f3be5e97640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3be0070b90 0x7f3be0205230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.868+0000 7f3be5e97640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3be00caed0 0x7f3be015e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.869+0000 7f3be5e97640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3be01581b0 0x7f3be015a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.869+0000 7f3be4e95640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3be0070b90 0x7f3be0205230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:25.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.869+0000 7f3be4e95640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3be0070b90 0x7f3be0205230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50838/0 (socket says 172.21.15.87:50838) 2024-09-17T03:14:25.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.869+0000 7f3be4e95640 1 -- 172.21.15.87:0/3126516154 learned_addr learned my addr 172.21.15.87:0/3126516154 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:25.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.869+0000 7f3be5e97640 1 -- 172.21.15.87:0/3126516154 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3be006ed60 con 0x7f3be0070b90 2024-09-17T03:14:25.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.869+0000 7f3be5e97640 1 -- 172.21.15.87:0/3126516154 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3be00cf6f0 con 0x7f3be00caed0 2024-09-17T03:14:25.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.869+0000 7f3be5e97640 1 -- 172.21.15.87:0/3126516154 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3be006eee0 con 0x7f3be01581b0 2024-09-17T03:14:25.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.869+0000 7f3bdffff640 1 --2- 172.21.15.87:0/3126516154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3be00caed0 0x7f3be015e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:25.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.870+0000 7f3be4e95640 1 -- 172.21.15.87:0/3126516154 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3be01581b0 msgr2=0x7f3be015a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:25.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.870+0000 7f3be4e95640 1 --2- 172.21.15.87:0/3126516154 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3be01581b0 0x7f3be015a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.870+0000 7f3be4e95640 1 -- 172.21.15.87:0/3126516154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3be00caed0 msgr2=0x7f3be015e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:25.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.870+0000 7f3be4e95640 1 --2- 172.21.15.87:0/3126516154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3be00caed0 0x7f3be015e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.870+0000 7f3be4e95640 1 -- 172.21.15.87:0/3126516154 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be00ca130 con 0x7f3be0070b90 2024-09-17T03:14:25.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.870+0000 7f3be4e95640 1 --2- 172.21.15.87:0/3126516154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3be0070b90 0x7f3be0205230 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400c3c0 tx=0x7f3bd400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:25.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.871+0000 7f16ad2b6640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16a80700d0 0x7f16a81559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.871+0000 7f16ad2b6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a80aee70 0x7f16a815bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.871+0000 7f16ad2b6640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f16a80c5ec0 0x7f16a815c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.866+0000 7fe16eb19640 1 Processor -- start 2024-09-17T03:14:25.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.866+0000 7fe16eb19640 1 -- start start 2024-09-17T03:14:25.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.868+0000 7fe16eb19640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe16806d320 0x7fe1681517b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.868+0000 7fe16eb19640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe168074e40 0x7fe1680acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.869+0000 7fe167fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe168074e40 0x7fe1680acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:25.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.869+0000 7fe167fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe168074e40 0x7fe1680acfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50834/0 (socket says 172.21.15.87:50834) 2024-09-17T03:14:25.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.869+0000 7fe16c8b5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe16806d320 0x7fe1681517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:25.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.869+0000 7fe16c8b5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe16806d320 0x7fe1681517b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.87:58900/0 (socket says 172.21.15.87:58900) 2024-09-17T03:14:25.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.869+0000 7fe167fff640 1 -- 172.21.15.87:0/2617470043 learned_addr learned my addr 172.21.15.87:0/2617470043 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:25.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.869+0000 7fe16eb19640 1 --2- 172.21.15.87:0/2617470043 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1681558c0 0x7fe16814fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.869+0000 7fe16eb19640 1 -- 172.21.15.87:0/2617470043 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe1680b5470 con 0x7fe168074e40 2024-09-17T03:14:25.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.869+0000 7fe16eb19640 1 -- 172.21.15.87:0/2617470043 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe1680b52f0 con 0x7fe16806d320 2024-09-17T03:14:25.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.869+0000 7fe16eb19640 1 -- 172.21.15.87:0/2617470043 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe1680b55f0 con 0x7fe1681558c0 2024-09-17T03:14:25.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.870+0000 7fe167fff640 1 -- 172.21.15.87:0/2617470043 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1681558c0 msgr2=0x7fe16814fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:25.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.870+0000 7fe167fff640 1 --2- 172.21.15.87:0/2617470043 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1681558c0 0x7fe16814fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.870+0000 7fe167fff640 1 -- 172.21.15.87:0/2617470043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe16806d320 msgr2=0x7fe1681517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:25.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.870+0000 7fe167fff640 1 --2- 172.21.15.87:0/2617470043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe16806d320 0x7fe1681517b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.870+0000 7fe167fff640 1 -- 172.21.15.87:0/2617470043 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe168151cf0 con 0x7fe168074e40 2024-09-17T03:14:25.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.870+0000 7fe167fff640 1 --2- 172.21.15.87:0/2617470043 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe168074e40 0x7fe1680acfa0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe15c01fd40 tx=0x7fe15c01d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:25.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.871+0000 7fe1657fa640 1 -- 172.21.15.87:0/2617470043 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c01ddc0 con 0x7fe168074e40 2024-09-17T03:14:25.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.871+0000 7f16ad2b6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f16a8168d80 con 0x7f16a80aee70 2024-09-17T03:14:25.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.871+0000 7f16ad2b6640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f16a8168c00 con 0x7f16a80700d0 2024-09-17T03:14:25.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.871+0000 7f16ad2b6640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f16a8168f00 con 0x7f16a80c5ec0 2024-09-17T03:14:25.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.871+0000 7fe16c8b5640 1 --2- 172.21.15.87:0/2617470043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe16806d320 0x7fe1681517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:14:25.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.871+0000 7f16a67fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a80aee70 0x7f16a815bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:25.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.871+0000 7f16a67fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a80aee70 0x7f16a815bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50842/0 (socket says 172.21.15.87:50842) 2024-09-17T03:14:25.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.871+0000 7f16a67fc640 1 -- 172.21.15.87:0/1751779568 learned_addr learned my addr 172.21.15.87:0/1751779568 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:25.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.871+0000 7f16a6ffd640 1 --2- 172.21.15.87:0/1751779568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16a80700d0 0x7f16a81559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:25.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.871+0000 7f16a77fe640 1 --2- 172.21.15.87:0/1751779568 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f16a80c5ec0 0x7f16a815c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:25.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.871+0000 7fe16eb19640 1 -- 172.21.15.87:0/2617470043 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1680ad4e0 con 0x7fe168074e40 2024-09-17T03:14:25.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.871+0000 7fe16eb19640 1 -- 172.21.15.87:0/2617470043 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe168153290 con 0x7fe168074e40 2024-09-17T03:14:25.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.871+0000 7fe1657fa640 1 -- 172.21.15.87:0/2617470043 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe15c004020 con 0x7fe168074e40 2024-09-17T03:14:25.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.871+0000 7fe1657fa640 1 -- 172.21.15.87:0/2617470043 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c004470 con 0x7fe168074e40 2024-09-17T03:14:25.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.872+0000 7f3bdd7fa640 1 -- 172.21.15.87:0/3126516154 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4011790 con 0x7f3be0070b90 2024-09-17T03:14:25.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.872+0000 7f3bdd7fa640 1 -- 172.21.15.87:0/3126516154 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bd4017070 con 0x7f3be0070b90 2024-09-17T03:14:25.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.872+0000 7f3be5e97640 1 -- 172.21.15.87:0/3126516154 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be00c95b0 con 0x7f3be0070b90 2024-09-17T03:14:25.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.872+0000 7f3be5e97640 1 -- 172.21.15.87:0/3126516154 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be00c9b70 con 0x7f3be0070b90 2024-09-17T03:14:25.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.873+0000 7f16a67fc640 1 -- 172.21.15.87:0/1751779568 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f16a80c5ec0 msgr2=0x7f16a815c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:25.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.873+0000 7f3bdd7fa640 1 -- 172.21.15.87:0/3126516154 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd4011cc0 con 0x7f3be0070b90 2024-09-17T03:14:25.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.873+0000 7fe16eb19640 1 -- 172.21.15.87:0/2617470043 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe148005290 con 0x7fe168074e40 2024-09-17T03:14:25.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.874+0000 7f16a67fc640 1 --2- 172.21.15.87:0/1751779568 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f16a80c5ec0 0x7f16a815c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.874+0000 7f16a67fc640 1 -- 172.21.15.87:0/1751779568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16a80700d0 msgr2=0x7f16a81559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:25.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.874+0000 7f16a67fc640 1 --2- 172.21.15.87:0/1751779568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16a80700d0 0x7f16a81559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:25.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.874+0000 7f3be5e97640 1 -- 172.21.15.87:0/3126516154 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bc0005290 con 0x7f3be0070b90 2024-09-17T03:14:25.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.874+0000 7f16a67fc640 1 -- 172.21.15.87:0/1751779568 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a8155f20 con 0x7f16a80aee70 2024-09-17T03:14:25.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.875+0000 7f16a6ffd640 1 --2- 172.21.15.87:0/1751779568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16a80700d0 0x7f16a81559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:25.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.875+0000 7f16a77fe640 1 --2- 172.21.15.87:0/1751779568 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f16a80c5ec0 0x7f16a815c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:14:25.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.876+0000 7f16a67fc640 1 --2- 172.21.15.87:0/1751779568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a80aee70 0x7f16a815bb10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f16a000eb60 tx=0x7f16a000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:25.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.876+0000 7f168f7fe640 1 -- 172.21.15.87:0/1751779568 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0010630 con 0x7f16a80aee70 2024-09-17T03:14:25.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.876+0000 7f168f7fe640 1 -- 172.21.15.87:0/1751779568 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16a00108c0 con 0x7f16a80aee70 2024-09-17T03:14:25.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.876+0000 7f16ad2b6640 1 -- 172.21.15.87:0/1751779568 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a80b5850 con 0x7f16a80aee70 2024-09-17T03:14:25.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.877+0000 7f16ad2b6640 1 -- 172.21.15.87:0/1751779568 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a8157da0 con 0x7f16a80aee70 2024-09-17T03:14:25.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.876+0000 7fe1657fa640 1 -- 172.21.15.87:0/2617470043 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe15c0035f0 con 0x7fe168074e40 2024-09-17T03:14:25.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.877+0000 7fe1657fa640 1 --2- 172.21.15.87:0/2617470043 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe14c078c90 0x7fe14c07b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.877+0000 7f168f7fe640 1 -- 172.21.15.87:0/1751779568 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0010d10 con 0x7f16a80aee70 2024-09-17T03:14:25.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.877+0000 7fe1657fa640 1 -- 172.21.15.87:0/2617470043 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe15c0bd670 con 0x7fe168074e40 2024-09-17T03:14:25.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.877+0000 7fe16c8b5640 1 --2- 172.21.15.87:0/2617470043 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe14c078c90 0x7fe14c07b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:25.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.878+0000 7f3bdd7fa640 1 -- 172.21.15.87:0/3126516154 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3bd401c460 con 0x7f3be0070b90 2024-09-17T03:14:25.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.879+0000 7fe16c8b5640 1 --2- 172.21.15.87:0/2617470043 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe14c078c90 0x7fe14c07b1a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe15400ba00 tx=0x7fe154005c80 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.879+0000 7f3bdd7fa640 1 --2- 172.21.15.87:0/3126516154 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3bcc078c90 0x7f3bcc07b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.879+0000 7f3bdd7fa640 1 -- 172.21.15.87:0/3126516154 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f3bd409a950 con 0x7f3be0070b90 2024-09-17T03:14:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.879+0000 7f3bdffff640 1 --2- 172.21.15.87:0/3126516154 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3bcc078c90 0x7f3bcc07b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:25.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.880+0000 7f16ad2b6640 1 -- 172.21.15.87:0/1751779568 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1688005290 con 0x7f16a80aee70 2024-09-17T03:14:25.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.880+0000 7f3bdffff640 1 --2- 172.21.15.87:0/3126516154 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3bcc078c90 0x7f3bcc07b1a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3be00cfbe0 tx=0x7f3bd00072f0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:25.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.887+0000 7fe1657fa640 1 -- 172.21.15.87:0/2617470043 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe15c085bb0 con 0x7fe168074e40 2024-09-17T03:14:25.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.889+0000 7f168f7fe640 1 -- 172.21.15.87:0/1751779568 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f16a0018420 con 0x7f16a80aee70 2024-09-17T03:14:25.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.890+0000 7f168f7fe640 1 --2- 172.21.15.87:0/1751779568 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1694078c10 0x7f169407b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:25.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.891+0000 7f16a6ffd640 1 --2- 172.21.15.87:0/1751779568 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1694078c10 0x7f169407b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:25.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.891+0000 7f16a6ffd640 1 --2- 172.21.15.87:0/1751779568 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1694078c10 0x7f169407b120 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1698001910 tx=0x7f1698009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:25.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.898+0000 7f3bdd7fa640 1 -- 172.21.15.87:0/3126516154 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3bd4062e00 con 0x7f3be0070b90 2024-09-17T03:14:25.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.902+0000 7f168f7fe640 1 -- 172.21.15.87:0/1751779568 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f16a00601e0 con 0x7f16a80aee70 2024-09-17T03:14:25.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:25.907+0000 7f168f7fe640 1 -- 172.21.15.87:0/1751779568 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f16a000ee60 con 0x7f16a80aee70 2024-09-17T03:14:26.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.179+0000 7fceb1c30640 1 -- 172.21.15.87:0/3686307917 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fcea40063d0 con 0x7fcea400ae50 2024-09-17T03:14:26.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.180+0000 7fce97fff640 1 -- 172.21.15.87:0/3686307917 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcea0084bf0 con 0x7fcea400ae50 2024-09-17T03:14:26.178 INFO:teuthology.orchestra.run.smithi087.stdout:55834574856 2024-09-17T03:14:26.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.181+0000 7f393b119640 1 -- 172.21.15.87:0/3413325153 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f39340ccfb0 con 0x7f39340cbb00 2024-09-17T03:14:26.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.182+0000 7f39317fa640 1 -- 172.21.15.87:0/3413325153 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f39280629f0 con 0x7f39340cbb00 2024-09-17T03:14:26.181 INFO:teuthology.orchestra.run.smithi087.stdout:64424509448 2024-09-17T03:14:26.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.183+0000 7fce95ffb640 1 -- 172.21.15.87:0/3686307917 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fce90078d80 msgr2=0x7fce9007b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.183+0000 7fce95ffb640 1 --2- 172.21.15.87:0/3686307917 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fce90078d80 0x7fce9007b290 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fce98000f60 tx=0x7fce98001510 comp rx=0 tx=0).stop 2024-09-17T03:14:26.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.183+0000 7fce95ffb640 1 -- 172.21.15.87:0/3686307917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcea400ae50 msgr2=0x7fcea417e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.183+0000 7fce95ffb640 1 --2- 172.21.15.87:0/3686307917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcea400ae50 0x7fcea417e840 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fcea002bdc0 tx=0x7fcea0030c60 comp rx=0 tx=0).stop 2024-09-17T03:14:26.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.184+0000 7fce95ffb640 1 -- 172.21.15.87:0/3686307917 shutdown_connections 2024-09-17T03:14:26.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.184+0000 7fce95ffb640 1 --2- 172.21.15.87:0/3686307917 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fce90078d80 0x7fce9007b290 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.184+0000 7fce95ffb640 1 --2- 172.21.15.87:0/3686307917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcea40e35c0 0x7fcea40db600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.184+0000 7fce95ffb640 1 --2- 172.21.15.87:0/3686307917 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcea4053c00 0x7fcea40dc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.184+0000 7fce95ffb640 1 --2- 172.21.15.87:0/3686307917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcea400ae50 0x7fcea417e840 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.184+0000 7fce95ffb640 1 -- 172.21.15.87:0/3686307917 >> 172.21.15.87:0/3686307917 conn(0x7fcea4051420 msgr2=0x7fcea400b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:26.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.184+0000 7fce95ffb640 1 -- 172.21.15.87:0/3686307917 shutdown_connections 2024-09-17T03:14:26.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.184+0000 7fce95ffb640 1 -- 172.21.15.87:0/3686307917 wait complete. 2024-09-17T03:14:26.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.185+0000 7f391affd640 1 -- 172.21.15.87:0/3413325153 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3920078c90 msgr2=0x7f392007b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.186+0000 7f391affd640 1 --2- 172.21.15.87:0/3413325153 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3920078c90 0x7f392007b1a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3924000f60 tx=0x7f3924009040 comp rx=0 tx=0).stop 2024-09-17T03:14:26.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.186+0000 7f391affd640 1 -- 172.21.15.87:0/3413325153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39340cbb00 msgr2=0x7f393415ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.186+0000 7f391affd640 1 --2- 172.21.15.87:0/3413325153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39340cbb00 0x7f393415ace0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f392800b780 tx=0x7f392800bc40 comp rx=0 tx=0).stop 2024-09-17T03:14:26.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.186+0000 7f391affd640 1 -- 172.21.15.87:0/3413325153 shutdown_connections 2024-09-17T03:14:26.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.186+0000 7f391affd640 1 --2- 172.21.15.87:0/3413325153 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3920078c90 0x7f392007b1a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.186+0000 7f391affd640 1 --2- 172.21.15.87:0/3413325153 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3934165a20 0x7f393415b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.186+0000 7f391affd640 1 --2- 172.21.15.87:0/3413325153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39340cbb00 0x7f393415ace0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.186+0000 7f391affd640 1 --2- 172.21.15.87:0/3413325153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f39340cb6f0 0x7f3934202a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.187+0000 7f391affd640 1 -- 172.21.15.87:0/3413325153 >> 172.21.15.87:0/3413325153 conn(0x7f393406ee70 msgr2=0x7f39340b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:26.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.187+0000 7f391affd640 1 -- 172.21.15.87:0/3413325153 shutdown_connections 2024-09-17T03:14:26.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.187+0000 7f391affd640 1 -- 172.21.15.87:0/3413325153 wait complete. 2024-09-17T03:14:26.201 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574856 for osd.4 2024-09-17T03:14:26.201 DEBUG:teuthology.parallel:result is None 2024-09-17T03:14:26.203 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.5 2024-09-17T03:14:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.204+0000 7fcadadda640 1 -- 172.21.15.87:0/895224512 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fcad40d0d70 con 0x7fcad414adc0 2024-09-17T03:14:26.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.204+0000 7fcad17fa640 1 -- 172.21.15.87:0/895224512 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcacc00ee60 con 0x7fcad414adc0 2024-09-17T03:14:26.203 DEBUG:teuthology.parallel:result is None 2024-09-17T03:14:26.204 INFO:teuthology.orchestra.run.smithi087.stdout:64424509449 2024-09-17T03:14:26.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.206+0000 7fcadadda640 1 -- 172.21.15.87:0/895224512 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcab8078b90 msgr2=0x7fcab807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.206+0000 7fcadadda640 1 --2- 172.21.15.87:0/895224512 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcab8078b90 0x7fcab807b0a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcac8005fd0 tx=0x7fcac800b3c0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.206+0000 7fcadadda640 1 -- 172.21.15.87:0/895224512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcad414adc0 msgr2=0x7fcad414b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.206+0000 7fcadadda640 1 --2- 172.21.15.87:0/895224512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcad414adc0 0x7fcad414b860 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcad4071c10 tx=0x7fcacc007720 comp rx=0 tx=0).stop 2024-09-17T03:14:26.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.207+0000 7fcadadda640 1 -- 172.21.15.87:0/895224512 shutdown_connections 2024-09-17T03:14:26.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.207+0000 7fcadadda640 1 --2- 172.21.15.87:0/895224512 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcab8078b90 0x7fcab807b0a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.207+0000 7fcadadda640 1 --2- 172.21.15.87:0/895224512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcad414adc0 0x7fcad414b860 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.207+0000 7fcadadda640 1 --2- 172.21.15.87:0/895224512 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcad414a9b0 0x7fcad414b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.207+0000 7fcadadda640 1 --2- 172.21.15.87:0/895224512 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcac4004800 0x7fcad41e9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.207+0000 7fcadadda640 1 -- 172.21.15.87:0/895224512 >> 172.21.15.87:0/895224512 conn(0x7fcad40d08c0 msgr2=0x7fcad415d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:26.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.207+0000 7fcadadda640 1 -- 172.21.15.87:0/895224512 shutdown_connections 2024-09-17T03:14:26.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.207+0000 7fcadadda640 1 -- 172.21.15.87:0/895224512 wait complete. 2024-09-17T03:14:26.221 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.6 2024-09-17T03:14:26.222 DEBUG:teuthology.parallel:result is None 2024-09-17T03:14:26.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.230+0000 7fdd01558640 1 -- 172.21.15.87:0/27768478 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fdcf4005c60 con 0x7fdcf4181540 2024-09-17T03:14:26.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.231+0000 7fdcf8ff9640 1 -- 172.21.15.87:0/27768478 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdce80738b0 con 0x7fdcf4181540 2024-09-17T03:14:26.230 INFO:teuthology.orchestra.run.smithi087.stdout:64424509449 2024-09-17T03:14:26.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.233+0000 7fa6f0ff9640 1 -- 172.21.15.87:0/2462460315 --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa7040b32c0 con 0x7fa704154080 2024-09-17T03:14:26.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.234+0000 7fdd01558640 1 -- 172.21.15.87:0/27768478 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdce0078b70 msgr2=0x7fdce007b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.234+0000 7fdd01558640 1 --2- 172.21.15.87:0/27768478 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdce0078b70 0x7fdce007b080 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000c2e0 tx=0x7fdcf0004240 comp rx=0 tx=0).stop 2024-09-17T03:14:26.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.234+0000 7fdd01558640 1 -- 172.21.15.87:0/27768478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcf4181540 msgr2=0x7fdcf4181950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.234+0000 7fdd01558640 1 --2- 172.21.15.87:0/27768478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcf4181540 0x7fdcf4181950 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdce800d780 tx=0x7fdce800dc40 comp rx=0 tx=0).stop 2024-09-17T03:14:26.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.234+0000 7fa6f2ffd640 1 -- 172.21.15.87:0/2462460315 <== mon.1 v2:172.21.15.149:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa6f8063ac0 con 0x7fa704154080 2024-09-17T03:14:26.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.234+0000 7fdd01558640 1 -- 172.21.15.87:0/27768478 shutdown_connections 2024-09-17T03:14:26.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.234+0000 7fdd01558640 1 --2- 172.21.15.87:0/27768478 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdce0078b70 0x7fdce007b080 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.234+0000 7fdd01558640 1 --2- 172.21.15.87:0/27768478 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdcf4181e90 0x7fdcf416d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.234+0000 7fdd01558640 1 --2- 172.21.15.87:0/27768478 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcf4181540 0x7fdcf4181950 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.234+0000 7fdd01558640 1 --2- 172.21.15.87:0/27768478 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdcec003a00 0x7fdcf40e36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.234+0000 7fdd01558640 1 -- 172.21.15.87:0/27768478 >> 172.21.15.87:0/27768478 conn(0x7fdcf4051420 msgr2=0x7fdcf40e6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:26.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.234+0000 7fdd01558640 1 -- 172.21.15.87:0/27768478 shutdown_connections 2024-09-17T03:14:26.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.234+0000 7fdd01558640 1 -- 172.21.15.87:0/27768478 wait complete. 2024-09-17T03:14:26.234 INFO:teuthology.orchestra.run.smithi087.stdout:64424509449 2024-09-17T03:14:26.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.237+0000 7fa6f0ff9640 1 -- 172.21.15.87:0/2462460315 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa6e4078c60 msgr2=0x7fa6e407b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.237+0000 7fa6f0ff9640 1 --2- 172.21.15.87:0/2462460315 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa6e4078c60 0x7fa6e407b170 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc034000 tx=0x7fa6fc033040 comp rx=0 tx=0).stop 2024-09-17T03:14:26.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.237+0000 7fa6f0ff9640 1 -- 172.21.15.87:0/2462460315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa704154080 msgr2=0x7fa70413ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.237+0000 7fa6f0ff9640 1 --2- 172.21.15.87:0/2462460315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa704154080 0x7fa70413ca90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa6f800e6d0 tx=0x7fa6f800eb90 comp rx=0 tx=0).stop 2024-09-17T03:14:26.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.237+0000 7fa6f0ff9640 1 -- 172.21.15.87:0/2462460315 shutdown_connections 2024-09-17T03:14:26.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.237+0000 7fa6f0ff9640 1 --2- 172.21.15.87:0/2462460315 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa6e4078c60 0x7fa6e407b170 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.237+0000 7fa6f0ff9640 1 --2- 172.21.15.87:0/2462460315 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa704154080 0x7fa70413ca90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.237+0000 7fa6f0ff9640 1 --2- 172.21.15.87:0/2462460315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa704153c70 0x7fa70413c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.237+0000 7fa6f0ff9640 1 --2- 172.21.15.87:0/2462460315 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa704155ab0 0x7fa704139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.237+0000 7fa6f0ff9640 1 -- 172.21.15.87:0/2462460315 >> 172.21.15.87:0/2462460315 conn(0x7fa7040b2d10 msgr2=0x7fa70415a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:26.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.237+0000 7fa6f0ff9640 1 -- 172.21.15.87:0/2462460315 shutdown_connections 2024-09-17T03:14:26.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.237+0000 7fa6f0ff9640 1 -- 172.21.15.87:0/2462460315 wait complete. 2024-09-17T03:14:26.245 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.7 2024-09-17T03:14:26.245 DEBUG:teuthology.parallel:result is None 2024-09-17T03:14:26.253 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.2 2024-09-17T03:14:26.253 DEBUG:teuthology.parallel:result is None 2024-09-17T03:14:26.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.259+0000 7fe16eb19640 1 -- 172.21.15.87:0/2617470043 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe148005580 con 0x7fe168074e40 2024-09-17T03:14:26.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.259+0000 7fe1657fa640 1 -- 172.21.15.87:0/2617470043 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe15c0854b0 con 0x7fe168074e40 2024-09-17T03:14:26.258 INFO:teuthology.orchestra.run.smithi087.stdout:64424509449 2024-09-17T03:14:26.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.261+0000 7fe16eb19640 1 -- 172.21.15.87:0/2617470043 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe14c078c90 msgr2=0x7fe14c07b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.261+0000 7fe16eb19640 1 --2- 172.21.15.87:0/2617470043 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe14c078c90 0x7fe14c07b1a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe15400ba00 tx=0x7fe154005c80 comp rx=0 tx=0).stop 2024-09-17T03:14:26.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.261+0000 7fe16eb19640 1 -- 172.21.15.87:0/2617470043 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe168074e40 msgr2=0x7fe1680acfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.261+0000 7fe16eb19640 1 --2- 172.21.15.87:0/2617470043 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe168074e40 0x7fe1680acfa0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe15c01fd40 tx=0x7fe15c01d6b0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.261+0000 7fe16eb19640 1 -- 172.21.15.87:0/2617470043 shutdown_connections 2024-09-17T03:14:26.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.261+0000 7fe16eb19640 1 --2- 172.21.15.87:0/2617470043 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe14c078c90 0x7fe14c07b1a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.261+0000 7fe16eb19640 1 --2- 172.21.15.87:0/2617470043 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1681558c0 0x7fe16814fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.261+0000 7fe16eb19640 1 --2- 172.21.15.87:0/2617470043 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe168074e40 0x7fe1680acfa0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.261+0000 7fe16eb19640 1 --2- 172.21.15.87:0/2617470043 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe16806d320 0x7fe1681517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.261+0000 7fe16eb19640 1 -- 172.21.15.87:0/2617470043 >> 172.21.15.87:0/2617470043 conn(0x7fe1680b2d10 msgr2=0x7fe168153d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:26.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.261+0000 7fe16eb19640 1 -- 172.21.15.87:0/2617470043 shutdown_connections 2024-09-17T03:14:26.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.262+0000 7fe16eb19640 1 -- 172.21.15.87:0/2617470043 wait complete. 2024-09-17T03:14:26.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.269+0000 7f3be5e97640 1 -- 172.21.15.87:0/3126516154 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3bc0005580 con 0x7f3be0070b90 2024-09-17T03:14:26.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.270+0000 7f3bdd7fa640 1 -- 172.21.15.87:0/3126516154 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3bd4062700 con 0x7f3be0070b90 2024-09-17T03:14:26.269 INFO:teuthology.orchestra.run.smithi087.stdout:64424509448 2024-09-17T03:14:26.271 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.1 2024-09-17T03:14:26.272 DEBUG:teuthology.parallel:result is None 2024-09-17T03:14:26.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.272+0000 7f3be5e97640 1 -- 172.21.15.87:0/3126516154 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3bcc078c90 msgr2=0x7f3bcc07b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.272+0000 7f3be5e97640 1 --2- 172.21.15.87:0/3126516154 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3bcc078c90 0x7f3bcc07b1a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3be00cfbe0 tx=0x7f3bd00072f0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.272+0000 7f3be5e97640 1 -- 172.21.15.87:0/3126516154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3be0070b90 msgr2=0x7f3be0205230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.272+0000 7f3be5e97640 1 --2- 172.21.15.87:0/3126516154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3be0070b90 0x7f3be0205230 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400c3c0 tx=0x7f3bd400c880 comp rx=0 tx=0).stop 2024-09-17T03:14:26.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.272+0000 7f3be5e97640 1 -- 172.21.15.87:0/3126516154 shutdown_connections 2024-09-17T03:14:26.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.272+0000 7f3be5e97640 1 --2- 172.21.15.87:0/3126516154 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3bcc078c90 0x7f3bcc07b1a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.272+0000 7f3be5e97640 1 --2- 172.21.15.87:0/3126516154 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3be01581b0 0x7f3be015a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.272+0000 7f3be5e97640 1 --2- 172.21.15.87:0/3126516154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3be00caed0 0x7f3be015e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.272+0000 7f3be5e97640 1 --2- 172.21.15.87:0/3126516154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3be0070b90 0x7f3be0205230 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.272+0000 7f3be5e97640 1 -- 172.21.15.87:0/3126516154 >> 172.21.15.87:0/3126516154 conn(0x7f3be00b2290 msgr2=0x7f3be00aec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:26.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.272+0000 7f3be5e97640 1 -- 172.21.15.87:0/3126516154 shutdown_connections 2024-09-17T03:14:26.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.272+0000 7f3be5e97640 1 -- 172.21.15.87:0/3126516154 wait complete. 2024-09-17T03:14:26.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.275+0000 7f16ad2b6640 1 -- 172.21.15.87:0/1751779568 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f1688005850 con 0x7f16a80aee70 2024-09-17T03:14:26.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.275+0000 7f168f7fe640 1 -- 172.21.15.87:0/1751779568 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f16a0064670 con 0x7f16a80aee70 2024-09-17T03:14:26.274 INFO:teuthology.orchestra.run.smithi087.stdout:64424509449 2024-09-17T03:14:26.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.277+0000 7f16ad2b6640 1 -- 172.21.15.87:0/1751779568 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1694078c10 msgr2=0x7f169407b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.277+0000 7f16ad2b6640 1 --2- 172.21.15.87:0/1751779568 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1694078c10 0x7f169407b120 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1698001910 tx=0x7f1698009040 comp rx=0 tx=0).stop 2024-09-17T03:14:26.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.277+0000 7f16ad2b6640 1 -- 172.21.15.87:0/1751779568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a80aee70 msgr2=0x7f16a815bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.277+0000 7f16ad2b6640 1 --2- 172.21.15.87:0/1751779568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a80aee70 0x7f16a815bb10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f16a000eb60 tx=0x7f16a000c6a0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.277+0000 7f16ad2b6640 1 -- 172.21.15.87:0/1751779568 shutdown_connections 2024-09-17T03:14:26.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.277+0000 7f16ad2b6640 1 --2- 172.21.15.87:0/1751779568 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1694078c10 0x7f169407b120 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.277+0000 7f16ad2b6640 1 --2- 172.21.15.87:0/1751779568 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f16a80c5ec0 0x7f16a815c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.277+0000 7f16ad2b6640 1 --2- 172.21.15.87:0/1751779568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16a80aee70 0x7f16a815bb10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.277+0000 7f16ad2b6640 1 --2- 172.21.15.87:0/1751779568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16a80700d0 0x7f16a81559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.277+0000 7f16ad2b6640 1 -- 172.21.15.87:0/1751779568 >> 172.21.15.87:0/1751779568 conn(0x7f16a80d0040 msgr2=0x7f16a815df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:26.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.278+0000 7f16ad2b6640 1 -- 172.21.15.87:0/1751779568 shutdown_connections 2024-09-17T03:14:26.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.278+0000 7f16ad2b6640 1 -- 172.21.15.87:0/1751779568 wait complete. 2024-09-17T03:14:26.283 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509448 for osd.0 2024-09-17T03:14:26.283 DEBUG:teuthology.parallel:result is None 2024-09-17T03:14:26.287 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509449 got 64424509449 for osd.3 2024-09-17T03:14:26.287 DEBUG:teuthology.parallel:result is None 2024-09-17T03:14:26.287 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T03:14:26.288 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:14:26.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.440+0000 7f777b83f640 1 -- 172.21.15.87:0/1477542857 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77740d1240 msgr2=0x7f777415b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.440+0000 7f777b83f640 1 --2- 172.21.15.87:0/1477542857 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77740d1240 0x7f777415b550 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7764009830 tx=0x7f776402a4a0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.440+0000 7f777b83f640 1 -- 172.21.15.87:0/1477542857 shutdown_connections 2024-09-17T03:14:26.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.441+0000 7f777b83f640 1 --2- 172.21.15.87:0/1477542857 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77740d1240 0x7f777415b550 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.441+0000 7f777b83f640 1 -- 172.21.15.87:0/1477542857 >> 172.21.15.87:0/1477542857 conn(0x7f777406cfc0 msgr2=0x7f777406d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:26.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.441+0000 7f777b83f640 1 -- 172.21.15.87:0/1477542857 shutdown_connections 2024-09-17T03:14:26.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.441+0000 7f777b83f640 1 -- 172.21.15.87:0/1477542857 wait complete. 2024-09-17T03:14:26.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.442+0000 7f777b83f640 1 Processor -- start 2024-09-17T03:14:26.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.442+0000 7f777b83f640 1 -- start start 2024-09-17T03:14:26.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.444+0000 7f777b83f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7774071120 0x7f77740a9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:26.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.444+0000 7f777b83f640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f77740ce690 0x7f7774146ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:26.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.444+0000 7f777b83f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77741585d0 0x7f77741e6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:26.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.444+0000 7f777b83f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f77740700b0 con 0x7f77741585d0 2024-09-17T03:14:26.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.444+0000 7f777b83f640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f777406ff30 con 0x7f7774071120 2024-09-17T03:14:26.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.444+0000 7f777b83f640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f7774070230 con 0x7f77740ce690 2024-09-17T03:14:26.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.444+0000 7f7779ddc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77741585d0 0x7f77741e6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:26.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.444+0000 7f7779ddc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77741585d0 0x7f77741e6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50866/0 (socket says 172.21.15.87:50866) 2024-09-17T03:14:26.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.444+0000 7f7779ddc640 1 -- 172.21.15.87:0/2692169128 learned_addr learned my addr 172.21.15.87:0/2692169128 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:26.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.444+0000 7f7778dda640 1 --2- 172.21.15.87:0/2692169128 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f77740ce690 0x7f7774146ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:26.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.444+0000 7f77795db640 1 --2- 172.21.15.87:0/2692169128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7774071120 0x7f77740a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:26.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.444+0000 7f7779ddc640 1 -- 172.21.15.87:0/2692169128 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f77740ce690 msgr2=0x7f7774146ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.444+0000 7f7779ddc640 1 --2- 172.21.15.87:0/2692169128 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f77740ce690 0x7f7774146ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.444+0000 7f7779ddc640 1 -- 172.21.15.87:0/2692169128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7774071120 msgr2=0x7f77740a9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.444+0000 7f7779ddc640 1 --2- 172.21.15.87:0/2692169128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7774071120 0x7f77740a9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.444+0000 7f7779ddc640 1 -- 172.21.15.87:0/2692169128 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77740a9ce0 con 0x7f77741585d0 2024-09-17T03:14:26.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.444+0000 7f7778dda640 1 --2- 172.21.15.87:0/2692169128 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f77740ce690 0x7f7774146ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:26.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.444+0000 7f77795db640 1 --2- 172.21.15.87:0/2692169128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7774071120 0x7f77740a9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:26.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.444+0000 7f7779ddc640 1 --2- 172.21.15.87:0/2692169128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77741585d0 0x7f77741e6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f776c00c6e0 tx=0x7f776c00cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:26.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.444+0000 7f7769ffb640 1 -- 172.21.15.87:0/2692169128 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c003b70 con 0x7f77741585d0 2024-09-17T03:14:26.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.444+0000 7f7769ffb640 1 -- 172.21.15.87:0/2692169128 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f776c003d10 con 0x7f77741585d0 2024-09-17T03:14:26.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.444+0000 7f777b83f640 1 -- 172.21.15.87:0/2692169128 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77740a9fc0 con 0x7f77741585d0 2024-09-17T03:14:26.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.444+0000 7f7769ffb640 1 -- 172.21.15.87:0/2692169128 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c005330 con 0x7f77741585d0 2024-09-17T03:14:26.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.445+0000 7f777b83f640 1 -- 172.21.15.87:0/2692169128 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7774148c50 con 0x7f77741585d0 2024-09-17T03:14:26.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.446+0000 7f777b83f640 1 -- 172.21.15.87:0/2692169128 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f777415a160 con 0x7f77741585d0 2024-09-17T03:14:26.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.446+0000 7f7769ffb640 1 -- 172.21.15.87:0/2692169128 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f776c0054d0 con 0x7f77741585d0 2024-09-17T03:14:26.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.447+0000 7f7769ffb640 1 --2- 172.21.15.87:0/2692169128 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7758078b90 0x7f775807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:26.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.447+0000 7f7769ffb640 1 -- 172.21.15.87:0/2692169128 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f776c09aac0 con 0x7f77741585d0 2024-09-17T03:14:26.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.447+0000 7f77795db640 1 --2- 172.21.15.87:0/2692169128 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7758078b90 0x7f775807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:26.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.447+0000 7f77795db640 1 --2- 172.21.15.87:0/2692169128 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7758078b90 0x7f775807b0a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7770001090 tx=0x7f7770009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:26.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.451+0000 7f7769ffb640 1 -- 172.21.15.87:0/2692169128 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f777415a160 con 0x7f77741585d0 2024-09-17T03:14:26.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.627+0000 7f777b83f640 1 -- 172.21.15.87:0/2692169128 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7774148ef0 con 0x7f7758078b90 2024-09-17T03:14:26.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.632+0000 7f7769ffb640 1 -- 172.21.15.87:0/2692169128 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251821 (secure 0 0 0) 0x7f7774148ef0 con 0x7f7758078b90 2024-09-17T03:14:26.633 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:26.634 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-09-17T03:14:26.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.635+0000 7f777b83f640 1 -- 172.21.15.87:0/2692169128 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7758078b90 msgr2=0x7f775807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.635+0000 7f777b83f640 1 --2- 172.21.15.87:0/2692169128 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7758078b90 0x7f775807b0a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7770001090 tx=0x7f7770009040 comp rx=0 tx=0).stop 2024-09-17T03:14:26.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.635+0000 7f777b83f640 1 -- 172.21.15.87:0/2692169128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77741585d0 msgr2=0x7f77741e6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.635+0000 7f777b83f640 1 --2- 172.21.15.87:0/2692169128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77741585d0 0x7f77741e6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f776c00c6e0 tx=0x7f776c00cba0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.635+0000 7f777b83f640 1 -- 172.21.15.87:0/2692169128 shutdown_connections 2024-09-17T03:14:26.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.635+0000 7f777b83f640 1 --2- 172.21.15.87:0/2692169128 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7758078b90 0x7f775807b0a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.635+0000 7f777b83f640 1 --2- 172.21.15.87:0/2692169128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77741585d0 0x7f77741e6190 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.635+0000 7f777b83f640 1 --2- 172.21.15.87:0/2692169128 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f77740ce690 0x7f7774146ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.635+0000 7f777b83f640 1 --2- 172.21.15.87:0/2692169128 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7774071120 0x7f77740a9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.635+0000 7f777b83f640 1 -- 172.21.15.87:0/2692169128 >> 172.21.15.87:0/2692169128 conn(0x7f777406cfc0 msgr2=0x7f777415a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:26.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.635+0000 7f777b83f640 1 -- 172.21.15.87:0/2692169128 shutdown_connections 2024-09-17T03:14:26.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.635+0000 7f777b83f640 1 -- 172.21.15.87:0/2692169128 wait complete. 2024-09-17T03:14:26.645 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-09-17T03:14:25.419310+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219280,"kb_used_data":3048,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755440,"statfs":{"total":773094113280,"available":772869570560,"internally_reserved":0,"allocated":3121152,"data_stored":1842412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.087903"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332877+0000","last_change":"2024-09-17T03:14:14.350186+0000","last_active":"2024-09-17T03:14:15.332877+0000","last_peered":"2024-09-17T03:14:15.332877+0000","last_clean":"2024-09-17T03:14:15.332877+0000","last_became_active":"2024-09-17T03:14:14.349855+0000","last_became_peered":"2024-09-17T03:14:14.349855+0000","last_unstale":"2024-09-17T03:14:15.332877+0000","last_undegraded":"2024-09-17T03:14:15.332877+0000","last_fullsized":"2024-09-17T03:14:15.332877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:56:09.730649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334764+0000","last_change":"2024-09-17T03:14:14.343284+0000","last_active":"2024-09-17T03:14:15.334764+0000","last_peered":"2024-09-17T03:14:15.334764+0000","last_clean":"2024-09-17T03:14:15.334764+0000","last_became_active":"2024-09-17T03:14:14.337583+0000","last_became_peered":"2024-09-17T03:14:14.337583+0000","last_unstale":"2024-09-17T03:14:15.334764+0000","last_undegraded":"2024-09-17T03:14:15.334764+0000","last_fullsized":"2024-09-17T03:14:15.334764+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:06:49.022656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334676+0000","last_change":"2024-09-17T03:14:14.340288+0000","last_active":"2024-09-17T03:14:15.334676+0000","last_peered":"2024-09-17T03:14:15.334676+0000","last_clean":"2024-09-17T03:14:15.334676+0000","last_became_active":"2024-09-17T03:14:14.338799+0000","last_became_peered":"2024-09-17T03:14:14.338799+0000","last_unstale":"2024-09-17T03:14:15.334676+0000","last_undegraded":"2024-09-17T03:14:15.334676+0000","last_fullsized":"2024-09-17T03:14:15.334676+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:16:54.254992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391881+0000","last_change":"2024-09-17T03:14:14.333722+0000","last_active":"2024-09-17T03:14:16.391881+0000","last_peered":"2024-09-17T03:14:16.391881+0000","last_clean":"2024-09-17T03:14:16.391881+0000","last_became_active":"2024-09-17T03:14:14.333401+0000","last_became_peered":"2024-09-17T03:14:14.333401+0000","last_unstale":"2024-09-17T03:14:16.391881+0000","last_undegraded":"2024-09-17T03:14:16.391881+0000","last_fullsized":"2024-09-17T03:14:16.391881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:40.499497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333292+0000","last_change":"2024-09-17T03:14:14.334841+0000","last_active":"2024-09-17T03:14:15.333292+0000","last_peered":"2024-09-17T03:14:15.333292+0000","last_clean":"2024-09-17T03:14:15.333292+0000","last_became_active":"2024-09-17T03:14:14.334543+0000","last_became_peered":"2024-09-17T03:14:14.334543+0000","last_unstale":"2024-09-17T03:14:15.333292+0000","last_undegraded":"2024-09-17T03:14:15.333292+0000","last_fullsized":"2024-09-17T03:14:15.333292+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:51.158396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391311+0000","last_change":"2024-09-17T03:14:14.339302+0000","last_active":"2024-09-17T03:14:16.391311+0000","last_peered":"2024-09-17T03:14:16.391311+0000","last_clean":"2024-09-17T03:14:16.391311+0000","last_became_active":"2024-09-17T03:14:14.331863+0000","last_became_peered":"2024-09-17T03:14:14.331863+0000","last_unstale":"2024-09-17T03:14:16.391311+0000","last_undegraded":"2024-09-17T03:14:16.391311+0000","last_fullsized":"2024-09-17T03:14:16.391311+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:40.639682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332440+0000","last_change":"2024-09-17T03:14:14.340253+0000","last_active":"2024-09-17T03:14:15.332440+0000","last_peered":"2024-09-17T03:14:15.332440+0000","last_clean":"2024-09-17T03:14:15.332440+0000","last_became_active":"2024-09-17T03:14:14.339855+0000","last_became_peered":"2024-09-17T03:14:14.339855+0000","last_unstale":"2024-09-17T03:14:15.332440+0000","last_undegraded":"2024-09-17T03:14:15.332440+0000","last_fullsized":"2024-09-17T03:14:15.332440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:44:20.684218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334380+0000","last_change":"2024-09-17T03:14:14.333479+0000","last_active":"2024-09-17T03:14:15.334380+0000","last_peered":"2024-09-17T03:14:15.334380+0000","last_clean":"2024-09-17T03:14:15.334380+0000","last_became_active":"2024-09-17T03:14:14.330382+0000","last_became_peered":"2024-09-17T03:14:14.330382+0000","last_unstale":"2024-09-17T03:14:15.334380+0000","last_undegraded":"2024-09-17T03:14:15.334380+0000","last_fullsized":"2024-09-17T03:14:15.334380+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:21.969469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391159+0000","last_change":"2024-09-17T03:14:14.345634+0000","last_active":"2024-09-17T03:14:16.391159+0000","last_peered":"2024-09-17T03:14:16.391159+0000","last_clean":"2024-09-17T03:14:16.391159+0000","last_became_active":"2024-09-17T03:14:14.345309+0000","last_became_peered":"2024-09-17T03:14:14.345309+0000","last_unstale":"2024-09-17T03:14:16.391159+0000","last_undegraded":"2024-09-17T03:14:16.391159+0000","last_fullsized":"2024-09-17T03:14:16.391159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:09.963317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332555+0000","last_change":"2024-09-17T03:14:14.337246+0000","last_active":"2024-09-17T03:14:15.332555+0000","last_peered":"2024-09-17T03:14:15.332555+0000","last_clean":"2024-09-17T03:14:15.332555+0000","last_became_active":"2024-09-17T03:14:14.336920+0000","last_became_peered":"2024-09-17T03:14:14.336920+0000","last_unstale":"2024-09-17T03:14:15.332555+0000","last_undegraded":"2024-09-17T03:14:15.332555+0000","last_fullsized":"2024-09-17T03:14:15.332555+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:35.974141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332769+0000","last_change":"2024-09-17T03:14:14.349173+0000","last_active":"2024-09-17T03:14:15.332769+0000","last_peered":"2024-09-17T03:14:15.332769+0000","last_clean":"2024-09-17T03:14:15.332769+0000","last_became_active":"2024-09-17T03:14:14.348420+0000","last_became_peered":"2024-09-17T03:14:14.348420+0000","last_unstale":"2024-09-17T03:14:15.332769+0000","last_undegraded":"2024-09-17T03:14:15.332769+0000","last_fullsized":"2024-09-17T03:14:15.332769+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:09.339640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332966+0000","last_change":"2024-09-17T03:14:14.336719+0000","last_active":"2024-09-17T03:14:15.332966+0000","last_peered":"2024-09-17T03:14:15.332966+0000","last_clean":"2024-09-17T03:14:15.332966+0000","last_became_active":"2024-09-17T03:14:14.336239+0000","last_became_peered":"2024-09-17T03:14:14.336239+0000","last_unstale":"2024-09-17T03:14:15.332966+0000","last_undegraded":"2024-09-17T03:14:15.332966+0000","last_fullsized":"2024-09-17T03:14:15.332966+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:15:59.839348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391861+0000","last_change":"2024-09-17T03:14:14.340337+0000","last_active":"2024-09-17T03:14:16.391861+0000","last_peered":"2024-09-17T03:14:16.391861+0000","last_clean":"2024-09-17T03:14:16.391861+0000","last_became_active":"2024-09-17T03:14:14.340028+0000","last_became_peered":"2024-09-17T03:14:14.340028+0000","last_unstale":"2024-09-17T03:14:16.391861+0000","last_undegraded":"2024-09-17T03:14:16.391861+0000","last_fullsized":"2024-09-17T03:14:16.391861+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:35.806642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392067+0000","last_change":"2024-09-17T03:14:14.334276+0000","last_active":"2024-09-17T03:14:16.392067+0000","last_peered":"2024-09-17T03:14:16.392067+0000","last_clean":"2024-09-17T03:14:16.392067+0000","last_became_active":"2024-09-17T03:14:14.333764+0000","last_became_peered":"2024-09-17T03:14:14.333764+0000","last_unstale":"2024-09-17T03:14:16.392067+0000","last_undegraded":"2024-09-17T03:14:16.392067+0000","last_fullsized":"2024-09-17T03:14:16.392067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:57.801941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333421+0000","last_change":"2024-09-17T03:14:14.337055+0000","last_active":"2024-09-17T03:14:15.333421+0000","last_peered":"2024-09-17T03:14:15.333421+0000","last_clean":"2024-09-17T03:14:15.333421+0000","last_became_active":"2024-09-17T03:14:14.334416+0000","last_became_peered":"2024-09-17T03:14:14.334416+0000","last_unstale":"2024-09-17T03:14:15.333421+0000","last_undegraded":"2024-09-17T03:14:15.333421+0000","last_fullsized":"2024-09-17T03:14:15.333421+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:03.846825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332440+0000","last_change":"2024-09-17T03:14:14.346619+0000","last_active":"2024-09-17T03:14:15.332440+0000","last_peered":"2024-09-17T03:14:15.332440+0000","last_clean":"2024-09-17T03:14:15.332440+0000","last_became_active":"2024-09-17T03:14:14.346322+0000","last_became_peered":"2024-09-17T03:14:14.346322+0000","last_unstale":"2024-09-17T03:14:15.332440+0000","last_undegraded":"2024-09-17T03:14:15.332440+0000","last_fullsized":"2024-09-17T03:14:15.332440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:34:00.384055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334241+0000","last_change":"2024-09-17T03:14:14.343978+0000","last_active":"2024-09-17T03:14:15.334241+0000","last_peered":"2024-09-17T03:14:15.334241+0000","last_clean":"2024-09-17T03:14:15.334241+0000","last_became_active":"2024-09-17T03:14:14.343600+0000","last_became_peered":"2024-09-17T03:14:14.343600+0000","last_unstale":"2024-09-17T03:14:15.334241+0000","last_undegraded":"2024-09-17T03:14:15.334241+0000","last_fullsized":"2024-09-17T03:14:15.334241+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:25.368172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391936+0000","last_change":"2024-09-17T03:14:14.336744+0000","last_active":"2024-09-17T03:14:16.391936+0000","last_peered":"2024-09-17T03:14:16.391936+0000","last_clean":"2024-09-17T03:14:16.391936+0000","last_became_active":"2024-09-17T03:14:14.336428+0000","last_became_peered":"2024-09-17T03:14:14.336428+0000","last_unstale":"2024-09-17T03:14:16.391936+0000","last_undegraded":"2024-09-17T03:14:16.391936+0000","last_fullsized":"2024-09-17T03:14:16.391936+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:09.963317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334349+0000","last_change":"2024-09-17T03:14:14.348250+0000","last_active":"2024-09-17T03:14:15.334349+0000","last_peered":"2024-09-17T03:14:15.334349+0000","last_clean":"2024-09-17T03:14:15.334349+0000","last_became_active":"2024-09-17T03:14:14.347538+0000","last_became_peered":"2024-09-17T03:14:14.347538+0000","last_unstale":"2024-09-17T03:14:15.334349+0000","last_undegraded":"2024-09-17T03:14:15.334349+0000","last_fullsized":"2024-09-17T03:14:15.334349+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:46.734553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333193+0000","last_change":"2024-09-17T03:14:14.335802+0000","last_active":"2024-09-17T03:14:15.333193+0000","last_peered":"2024-09-17T03:14:15.333193+0000","last_clean":"2024-09-17T03:14:15.333193+0000","last_became_active":"2024-09-17T03:14:14.335507+0000","last_became_peered":"2024-09-17T03:14:14.335507+0000","last_unstale":"2024-09-17T03:14:15.333193+0000","last_undegraded":"2024-09-17T03:14:15.333193+0000","last_fullsized":"2024-09-17T03:14:15.333193+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:29.705538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333725+0000","last_change":"2024-09-17T03:14:14.338758+0000","last_active":"2024-09-17T03:14:15.333725+0000","last_peered":"2024-09-17T03:14:15.333725+0000","last_clean":"2024-09-17T03:14:15.333725+0000","last_became_active":"2024-09-17T03:14:14.337821+0000","last_became_peered":"2024-09-17T03:14:14.337821+0000","last_unstale":"2024-09-17T03:14:15.333725+0000","last_undegraded":"2024-09-17T03:14:15.333725+0000","last_fullsized":"2024-09-17T03:14:15.333725+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:09:24.347980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391355+0000","last_change":"2024-09-17T03:14:14.332615+0000","last_active":"2024-09-17T03:14:16.391355+0000","last_peered":"2024-09-17T03:14:16.391355+0000","last_clean":"2024-09-17T03:14:16.391355+0000","last_became_active":"2024-09-17T03:14:14.332285+0000","last_became_peered":"2024-09-17T03:14:14.332285+0000","last_unstale":"2024-09-17T03:14:16.391355+0000","last_undegraded":"2024-09-17T03:14:16.391355+0000","last_fullsized":"2024-09-17T03:14:16.391355+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:40.435687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333687+0000","last_change":"2024-09-17T03:14:14.342718+0000","last_active":"2024-09-17T03:14:15.333687+0000","last_peered":"2024-09-17T03:14:15.333687+0000","last_clean":"2024-09-17T03:14:15.333687+0000","last_became_active":"2024-09-17T03:14:14.341029+0000","last_became_peered":"2024-09-17T03:14:14.341029+0000","last_unstale":"2024-09-17T03:14:15.333687+0000","last_undegraded":"2024-09-17T03:14:15.333687+0000","last_fullsized":"2024-09-17T03:14:15.333687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:24:30.789181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332649+0000","last_change":"2024-09-17T03:14:14.344909+0000","last_active":"2024-09-17T03:14:15.332649+0000","last_peered":"2024-09-17T03:14:15.332649+0000","last_clean":"2024-09-17T03:14:15.332649+0000","last_became_active":"2024-09-17T03:14:14.343198+0000","last_became_peered":"2024-09-17T03:14:14.343198+0000","last_unstale":"2024-09-17T03:14:15.332649+0000","last_undegraded":"2024-09-17T03:14:15.332649+0000","last_fullsized":"2024-09-17T03:14:15.332649+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:06.251650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334992+0000","last_change":"2024-09-17T03:14:14.339599+0000","last_active":"2024-09-17T03:14:15.334992+0000","last_peered":"2024-09-17T03:14:15.334992+0000","last_clean":"2024-09-17T03:14:15.334992+0000","last_became_active":"2024-09-17T03:14:14.339248+0000","last_became_peered":"2024-09-17T03:14:14.339248+0000","last_unstale":"2024-09-17T03:14:15.334992+0000","last_undegraded":"2024-09-17T03:14:15.334992+0000","last_fullsized":"2024-09-17T03:14:15.334992+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:57:00.810832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392086+0000","last_change":"2024-09-17T03:14:14.347769+0000","last_active":"2024-09-17T03:14:16.392086+0000","last_peered":"2024-09-17T03:14:16.392086+0000","last_clean":"2024-09-17T03:14:16.392086+0000","last_became_active":"2024-09-17T03:14:14.347449+0000","last_became_peered":"2024-09-17T03:14:14.347449+0000","last_unstale":"2024-09-17T03:14:16.392086+0000","last_undegraded":"2024-09-17T03:14:16.392086+0000","last_fullsized":"2024-09-17T03:14:16.392086+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:14.698488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392056+0000","last_change":"2024-09-17T03:14:14.334291+0000","last_active":"2024-09-17T03:14:16.392056+0000","last_peered":"2024-09-17T03:14:16.392056+0000","last_clean":"2024-09-17T03:14:16.392056+0000","last_became_active":"2024-09-17T03:14:14.333993+0000","last_became_peered":"2024-09-17T03:14:14.333993+0000","last_unstale":"2024-09-17T03:14:16.392056+0000","last_undegraded":"2024-09-17T03:14:16.392056+0000","last_fullsized":"2024-09-17T03:14:16.392056+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:57.558924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392260+0000","last_change":"2024-09-17T03:14:14.344498+0000","last_active":"2024-09-17T03:14:16.392260+0000","last_peered":"2024-09-17T03:14:16.392260+0000","last_clean":"2024-09-17T03:14:16.392260+0000","last_became_active":"2024-09-17T03:14:14.344151+0000","last_became_peered":"2024-09-17T03:14:14.344151+0000","last_unstale":"2024-09-17T03:14:16.392260+0000","last_undegraded":"2024-09-17T03:14:16.392260+0000","last_fullsized":"2024-09-17T03:14:16.392260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:04:24.782583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334896+0000","last_change":"2024-09-17T03:14:14.327110+0000","last_active":"2024-09-17T03:14:15.334896+0000","last_peered":"2024-09-17T03:14:15.334896+0000","last_clean":"2024-09-17T03:14:15.334896+0000","last_became_active":"2024-09-17T03:14:14.326521+0000","last_became_peered":"2024-09-17T03:14:14.326521+0000","last_unstale":"2024-09-17T03:14:15.334896+0000","last_undegraded":"2024-09-17T03:14:15.334896+0000","last_fullsized":"2024-09-17T03:14:15.334896+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:42.804342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333824+0000","last_change":"2024-09-17T03:14:14.344297+0000","last_active":"2024-09-17T03:14:15.333824+0000","last_peered":"2024-09-17T03:14:15.333824+0000","last_clean":"2024-09-17T03:14:15.333824+0000","last_became_active":"2024-09-17T03:14:14.343941+0000","last_became_peered":"2024-09-17T03:14:14.343941+0000","last_unstale":"2024-09-17T03:14:15.333824+0000","last_undegraded":"2024-09-17T03:14:15.333824+0000","last_fullsized":"2024-09-17T03:14:15.333824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:18:47.095575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391581+0000","last_change":"2024-09-17T03:14:14.335330+0000","last_active":"2024-09-17T03:14:16.391581+0000","last_peered":"2024-09-17T03:14:16.391581+0000","last_clean":"2024-09-17T03:14:16.391581+0000","last_became_active":"2024-09-17T03:14:14.334989+0000","last_became_peered":"2024-09-17T03:14:14.334989+0000","last_unstale":"2024-09-17T03:14:16.391581+0000","last_undegraded":"2024-09-17T03:14:16.391581+0000","last_fullsized":"2024-09-17T03:14:16.391581+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:30.375807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391256+0000","last_change":"2024-09-17T03:14:14.334915+0000","last_active":"2024-09-17T03:14:16.391256+0000","last_peered":"2024-09-17T03:14:16.391256+0000","last_clean":"2024-09-17T03:14:16.391256+0000","last_became_active":"2024-09-17T03:14:14.334587+0000","last_became_peered":"2024-09-17T03:14:14.334587+0000","last_unstale":"2024-09-17T03:14:16.391256+0000","last_undegraded":"2024-09-17T03:14:16.391256+0000","last_fullsized":"2024-09-17T03:14:16.391256+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:55:20.376892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391571+0000","last_change":"2024-09-17T03:14:14.335030+0000","last_active":"2024-09-17T03:14:16.391571+0000","last_peered":"2024-09-17T03:14:16.391571+0000","last_clean":"2024-09-17T03:14:16.391571+0000","last_became_active":"2024-09-17T03:14:14.334722+0000","last_became_peered":"2024-09-17T03:14:14.334722+0000","last_unstale":"2024-09-17T03:14:16.391571+0000","last_undegraded":"2024-09-17T03:14:16.391571+0000","last_fullsized":"2024-09-17T03:14:16.391571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:59:38.941073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334406+0000","last_change":"2024-09-17T03:14:13.319793+0000","last_active":"2024-09-17T03:14:15.334406+0000","last_peered":"2024-09-17T03:14:15.334406+0000","last_clean":"2024-09-17T03:14:15.334406+0000","last_became_active":"2024-09-17T03:14:13.319467+0000","last_became_peered":"2024-09-17T03:14:13.319467+0000","last_unstale":"2024-09-17T03:14:15.334406+0000","last_undegraded":"2024-09-17T03:14:15.334406+0000","last_fullsized":"2024-09-17T03:14:15.334406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:06:54.724381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333960+0000","last_change":"2024-09-17T03:14:14.330562+0000","last_active":"2024-09-17T03:14:15.333960+0000","last_peered":"2024-09-17T03:14:15.333960+0000","last_clean":"2024-09-17T03:14:15.333960+0000","last_became_active":"2024-09-17T03:14:14.330255+0000","last_became_peered":"2024-09-17T03:14:14.330255+0000","last_unstale":"2024-09-17T03:14:15.333960+0000","last_undegraded":"2024-09-17T03:14:15.333960+0000","last_fullsized":"2024-09-17T03:14:15.333960+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:52:15.666625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333906+0000","last_change":"2024-09-17T03:14:13.324436+0000","last_active":"2024-09-17T03:14:15.333906+0000","last_peered":"2024-09-17T03:14:15.333906+0000","last_clean":"2024-09-17T03:14:15.333906+0000","last_became_active":"2024-09-17T03:14:13.323350+0000","last_became_peered":"2024-09-17T03:14:13.323350+0000","last_unstale":"2024-09-17T03:14:15.333906+0000","last_undegraded":"2024-09-17T03:14:15.333906+0000","last_fullsized":"2024-09-17T03:14:15.333906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:13.645071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391387+0000","last_change":"2024-09-17T03:14:13.330532+0000","last_active":"2024-09-17T03:14:16.391387+0000","last_peered":"2024-09-17T03:14:16.391387+0000","last_clean":"2024-09-17T03:14:16.391387+0000","last_became_active":"2024-09-17T03:14:13.330217+0000","last_became_peered":"2024-09-17T03:14:13.330217+0000","last_unstale":"2024-09-17T03:14:16.391387+0000","last_undegraded":"2024-09-17T03:14:16.391387+0000","last_fullsized":"2024-09-17T03:14:16.391387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:00.266996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334660+0000","last_change":"2024-09-17T03:14:14.337981+0000","last_active":"2024-09-17T03:14:15.334660+0000","last_peered":"2024-09-17T03:14:15.334660+0000","last_clean":"2024-09-17T03:14:15.334660+0000","last_became_active":"2024-09-17T03:14:14.336212+0000","last_became_peered":"2024-09-17T03:14:14.336212+0000","last_unstale":"2024-09-17T03:14:15.334660+0000","last_undegraded":"2024-09-17T03:14:15.334660+0000","last_fullsized":"2024-09-17T03:14:15.334660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:12:16.212384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332704+0000","last_change":"2024-09-17T03:14:13.316453+0000","last_active":"2024-09-17T03:14:15.332704+0000","last_peered":"2024-09-17T03:14:15.332704+0000","last_clean":"2024-09-17T03:14:15.332704+0000","last_became_active":"2024-09-17T03:14:13.316149+0000","last_became_peered":"2024-09-17T03:14:13.316149+0000","last_unstale":"2024-09-17T03:14:15.332704+0000","last_undegraded":"2024-09-17T03:14:15.332704+0000","last_fullsized":"2024-09-17T03:14:15.332704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:39.494722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334061+0000","last_change":"2024-09-17T03:14:14.339215+0000","last_active":"2024-09-17T03:14:15.334061+0000","last_peered":"2024-09-17T03:14:15.334061+0000","last_clean":"2024-09-17T03:14:15.334061+0000","last_became_active":"2024-09-17T03:14:14.337738+0000","last_became_peered":"2024-09-17T03:14:14.337738+0000","last_unstale":"2024-09-17T03:14:15.334061+0000","last_undegraded":"2024-09-17T03:14:15.334061+0000","last_fullsized":"2024-09-17T03:14:15.334061+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:18.278040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391218+0000","last_change":"2024-09-17T03:14:13.321965+0000","last_active":"2024-09-17T03:14:16.391218+0000","last_peered":"2024-09-17T03:14:16.391218+0000","last_clean":"2024-09-17T03:14:16.391218+0000","last_became_active":"2024-09-17T03:14:13.321616+0000","last_became_peered":"2024-09-17T03:14:13.321616+0000","last_unstale":"2024-09-17T03:14:16.391218+0000","last_undegraded":"2024-09-17T03:14:16.391218+0000","last_fullsized":"2024-09-17T03:14:16.391218+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:05:46.263336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392183+0000","last_change":"2024-09-17T03:14:14.348468+0000","last_active":"2024-09-17T03:14:16.392183+0000","last_peered":"2024-09-17T03:14:16.392183+0000","last_clean":"2024-09-17T03:14:16.392183+0000","last_became_active":"2024-09-17T03:14:14.348144+0000","last_became_peered":"2024-09-17T03:14:14.348144+0000","last_unstale":"2024-09-17T03:14:16.392183+0000","last_undegraded":"2024-09-17T03:14:16.392183+0000","last_fullsized":"2024-09-17T03:14:16.392183+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:14.649846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332338+0000","last_change":"2024-09-17T03:14:13.329841+0000","last_active":"2024-09-17T03:14:15.332338+0000","last_peered":"2024-09-17T03:14:15.332338+0000","last_clean":"2024-09-17T03:14:15.332338+0000","last_became_active":"2024-09-17T03:14:13.329154+0000","last_became_peered":"2024-09-17T03:14:13.329154+0000","last_unstale":"2024-09-17T03:14:15.332338+0000","last_undegraded":"2024-09-17T03:14:15.332338+0000","last_fullsized":"2024-09-17T03:14:15.332338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:39.634907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.335175+0000","last_change":"2024-09-17T03:14:14.345392+0000","last_active":"2024-09-17T03:14:15.335175+0000","last_peered":"2024-09-17T03:14:15.335175+0000","last_clean":"2024-09-17T03:14:15.335175+0000","last_became_active":"2024-09-17T03:14:14.345073+0000","last_became_peered":"2024-09-17T03:14:14.345073+0000","last_unstale":"2024-09-17T03:14:15.335175+0000","last_undegraded":"2024-09-17T03:14:15.335175+0000","last_fullsized":"2024-09-17T03:14:15.335175+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:04.528125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391685+0000","last_change":"2024-09-17T03:14:13.313377+0000","last_active":"2024-09-17T03:14:16.391685+0000","last_peered":"2024-09-17T03:14:16.391685+0000","last_clean":"2024-09-17T03:14:16.391685+0000","last_became_active":"2024-09-17T03:14:13.312848+0000","last_became_peered":"2024-09-17T03:14:13.312848+0000","last_unstale":"2024-09-17T03:14:16.391685+0000","last_undegraded":"2024-09-17T03:14:16.391685+0000","last_fullsized":"2024-09-17T03:14:16.391685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:00.266996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333208+0000","last_change":"2024-09-17T03:14:14.344920+0000","last_active":"2024-09-17T03:14:15.333208+0000","last_peered":"2024-09-17T03:14:15.333208+0000","last_clean":"2024-09-17T03:14:15.333208+0000","last_became_active":"2024-09-17T03:14:14.344570+0000","last_became_peered":"2024-09-17T03:14:14.344570+0000","last_unstale":"2024-09-17T03:14:15.333208+0000","last_undegraded":"2024-09-17T03:14:15.333208+0000","last_fullsized":"2024-09-17T03:14:15.333208+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:03.086199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391710+0000","last_change":"2024-09-17T03:14:13.322066+0000","last_active":"2024-09-17T03:14:16.391710+0000","last_peered":"2024-09-17T03:14:16.391710+0000","last_clean":"2024-09-17T03:14:16.391710+0000","last_became_active":"2024-09-17T03:14:13.321679+0000","last_became_peered":"2024-09-17T03:14:13.321679+0000","last_unstale":"2024-09-17T03:14:16.391710+0000","last_undegraded":"2024-09-17T03:14:16.391710+0000","last_fullsized":"2024-09-17T03:14:16.391710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:55.304571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333004+0000","last_change":"2024-09-17T03:14:14.337601+0000","last_active":"2024-09-17T03:14:15.333004+0000","last_peered":"2024-09-17T03:14:15.333004+0000","last_clean":"2024-09-17T03:14:15.333004+0000","last_became_active":"2024-09-17T03:14:14.336457+0000","last_became_peered":"2024-09-17T03:14:14.336457+0000","last_unstale":"2024-09-17T03:14:15.333004+0000","last_undegraded":"2024-09-17T03:14:15.333004+0000","last_fullsized":"2024-09-17T03:14:15.333004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:53:37.876227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391686+0000","last_change":"2024-09-17T03:14:13.318626+0000","last_active":"2024-09-17T03:14:16.391686+0000","last_peered":"2024-09-17T03:14:16.391686+0000","last_clean":"2024-09-17T03:14:16.391686+0000","last_became_active":"2024-09-17T03:14:13.318302+0000","last_became_peered":"2024-09-17T03:14:13.318302+0000","last_unstale":"2024-09-17T03:14:16.391686+0000","last_undegraded":"2024-09-17T03:14:16.391686+0000","last_fullsized":"2024-09-17T03:14:16.391686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:53.797340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392030+0000","last_change":"2024-09-17T03:14:14.344640+0000","last_active":"2024-09-17T03:14:16.392030+0000","last_peered":"2024-09-17T03:14:16.392030+0000","last_clean":"2024-09-17T03:14:16.392030+0000","last_became_active":"2024-09-17T03:14:14.344288+0000","last_became_peered":"2024-09-17T03:14:14.344288+0000","last_unstale":"2024-09-17T03:14:16.392030+0000","last_undegraded":"2024-09-17T03:14:16.392030+0000","last_fullsized":"2024-09-17T03:14:16.392030+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:55:42.826482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333916+0000","last_change":"2024-09-17T03:14:13.317055+0000","last_active":"2024-09-17T03:14:15.333916+0000","last_peered":"2024-09-17T03:14:15.333916+0000","last_clean":"2024-09-17T03:14:15.333916+0000","last_became_active":"2024-09-17T03:14:13.316736+0000","last_became_peered":"2024-09-17T03:14:13.316736+0000","last_unstale":"2024-09-17T03:14:15.333916+0000","last_undegraded":"2024-09-17T03:14:15.333916+0000","last_fullsized":"2024-09-17T03:14:15.333916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:13.543618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333254+0000","last_change":"2024-09-17T03:14:14.341872+0000","last_active":"2024-09-17T03:14:15.333254+0000","last_peered":"2024-09-17T03:14:15.333254+0000","last_clean":"2024-09-17T03:14:15.333254+0000","last_became_active":"2024-09-17T03:14:14.341555+0000","last_became_peered":"2024-09-17T03:14:14.341555+0000","last_unstale":"2024-09-17T03:14:15.333254+0000","last_undegraded":"2024-09-17T03:14:15.333254+0000","last_fullsized":"2024-09-17T03:14:15.333254+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:14.208665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332452+0000","last_change":"2024-09-17T03:14:13.325444+0000","last_active":"2024-09-17T03:14:15.332452+0000","last_peered":"2024-09-17T03:14:15.332452+0000","last_clean":"2024-09-17T03:14:15.332452+0000","last_became_active":"2024-09-17T03:14:13.325121+0000","last_became_peered":"2024-09-17T03:14:13.325121+0000","last_unstale":"2024-09-17T03:14:15.332452+0000","last_undegraded":"2024-09-17T03:14:15.332452+0000","last_fullsized":"2024-09-17T03:14:15.332452+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:55:19.372118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.390899+0000","last_change":"2024-09-17T03:14:14.341354+0000","last_active":"2024-09-17T03:14:16.390899+0000","last_peered":"2024-09-17T03:14:16.390899+0000","last_clean":"2024-09-17T03:14:16.390899+0000","last_became_active":"2024-09-17T03:14:14.341045+0000","last_became_peered":"2024-09-17T03:14:14.341045+0000","last_unstale":"2024-09-17T03:14:16.390899+0000","last_undegraded":"2024-09-17T03:14:16.390899+0000","last_fullsized":"2024-09-17T03:14:16.390899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:30.375807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333084+0000","last_change":"2024-09-17T03:14:13.321317+0000","last_active":"2024-09-17T03:14:15.333084+0000","last_peered":"2024-09-17T03:14:15.333084+0000","last_clean":"2024-09-17T03:14:15.333084+0000","last_became_active":"2024-09-17T03:14:13.320995+0000","last_became_peered":"2024-09-17T03:14:13.320995+0000","last_unstale":"2024-09-17T03:14:15.333084+0000","last_undegraded":"2024-09-17T03:14:15.333084+0000","last_fullsized":"2024-09-17T03:14:15.333084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:56.906875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333598+0000","last_change":"2024-09-17T03:14:14.342775+0000","last_active":"2024-09-17T03:14:15.333598+0000","last_peered":"2024-09-17T03:14:15.333598+0000","last_clean":"2024-09-17T03:14:15.333598+0000","last_became_active":"2024-09-17T03:14:14.342463+0000","last_became_peered":"2024-09-17T03:14:14.342463+0000","last_unstale":"2024-09-17T03:14:15.333598+0000","last_undegraded":"2024-09-17T03:14:15.333598+0000","last_fullsized":"2024-09-17T03:14:15.333598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:29:58.176940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333151+0000","last_change":"2024-09-17T03:14:13.324555+0000","last_active":"2024-09-17T03:14:15.333151+0000","last_peered":"2024-09-17T03:14:15.333151+0000","last_clean":"2024-09-17T03:14:15.333151+0000","last_became_active":"2024-09-17T03:14:13.323910+0000","last_became_peered":"2024-09-17T03:14:13.323910+0000","last_unstale":"2024-09-17T03:14:15.333151+0000","last_undegraded":"2024-09-17T03:14:15.333151+0000","last_fullsized":"2024-09-17T03:14:15.333151+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:48.923806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392232+0000","last_change":"2024-09-17T03:14:14.334852+0000","last_active":"2024-09-17T03:14:16.392232+0000","last_peered":"2024-09-17T03:14:16.392232+0000","last_clean":"2024-09-17T03:14:16.392232+0000","last_became_active":"2024-09-17T03:14:14.334469+0000","last_became_peered":"2024-09-17T03:14:14.334469+0000","last_unstale":"2024-09-17T03:14:16.392232+0000","last_undegraded":"2024-09-17T03:14:16.392232+0000","last_fullsized":"2024-09-17T03:14:16.392232+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:14.548392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333987+0000","last_change":"2024-09-17T03:14:13.322488+0000","last_active":"2024-09-17T03:14:15.333987+0000","last_peered":"2024-09-17T03:14:15.333987+0000","last_clean":"2024-09-17T03:14:15.333987+0000","last_became_active":"2024-09-17T03:14:13.322170+0000","last_became_peered":"2024-09-17T03:14:13.322170+0000","last_unstale":"2024-09-17T03:14:15.333987+0000","last_undegraded":"2024-09-17T03:14:15.333987+0000","last_fullsized":"2024-09-17T03:14:15.333987+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:13.679632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333094+0000","last_change":"2024-09-17T03:14:14.336281+0000","last_active":"2024-09-17T03:14:15.333094+0000","last_peered":"2024-09-17T03:14:15.333094+0000","last_clean":"2024-09-17T03:14:15.333094+0000","last_became_active":"2024-09-17T03:14:14.335986+0000","last_became_peered":"2024-09-17T03:14:14.335986+0000","last_unstale":"2024-09-17T03:14:15.333094+0000","last_undegraded":"2024-09-17T03:14:15.333094+0000","last_fullsized":"2024-09-17T03:14:15.333094+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:18.826069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334554+0000","last_change":"2024-09-17T03:14:13.318667+0000","last_active":"2024-09-17T03:14:15.334554+0000","last_peered":"2024-09-17T03:14:15.334554+0000","last_clean":"2024-09-17T03:14:15.334554+0000","last_became_active":"2024-09-17T03:14:13.318364+0000","last_became_peered":"2024-09-17T03:14:13.318364+0000","last_unstale":"2024-09-17T03:14:15.334554+0000","last_undegraded":"2024-09-17T03:14:15.334554+0000","last_fullsized":"2024-09-17T03:14:15.334554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:55.719231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333416+0000","last_change":"2024-09-17T03:14:14.334875+0000","last_active":"2024-09-17T03:14:15.333416+0000","last_peered":"2024-09-17T03:14:15.333416+0000","last_clean":"2024-09-17T03:14:15.333416+0000","last_became_active":"2024-09-17T03:14:14.334511+0000","last_became_peered":"2024-09-17T03:14:14.334511+0000","last_unstale":"2024-09-17T03:14:15.333416+0000","last_undegraded":"2024-09-17T03:14:15.333416+0000","last_fullsized":"2024-09-17T03:14:15.333416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:26.597012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392173+0000","last_change":"2024-09-17T03:14:13.330509+0000","last_active":"2024-09-17T03:14:16.392173+0000","last_peered":"2024-09-17T03:14:16.392173+0000","last_clean":"2024-09-17T03:14:16.392173+0000","last_became_active":"2024-09-17T03:14:13.330197+0000","last_became_peered":"2024-09-17T03:14:13.330197+0000","last_unstale":"2024-09-17T03:14:16.392173+0000","last_undegraded":"2024-09-17T03:14:16.392173+0000","last_fullsized":"2024-09-17T03:14:16.392173+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:53.797340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334097+0000","last_change":"2024-09-17T03:14:14.345943+0000","last_active":"2024-09-17T03:14:15.334097+0000","last_peered":"2024-09-17T03:14:15.334097+0000","last_clean":"2024-09-17T03:14:15.334097+0000","last_became_active":"2024-09-17T03:14:14.345637+0000","last_became_peered":"2024-09-17T03:14:14.345637+0000","last_unstale":"2024-09-17T03:14:15.334097+0000","last_undegraded":"2024-09-17T03:14:15.334097+0000","last_fullsized":"2024-09-17T03:14:15.334097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:23.070846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.390671+0000","last_change":"2024-09-17T03:14:04.862167+0000","last_active":"2024-09-17T03:14:16.390671+0000","last_peered":"2024-09-17T03:14:16.390671+0000","last_clean":"2024-09-17T03:14:16.390671+0000","last_became_active":"2024-09-17T03:14:04.861731+0000","last_became_peered":"2024-09-17T03:14:04.861731+0000","last_unstale":"2024-09-17T03:14:16.390671+0000","last_undegraded":"2024-09-17T03:14:16.390671+0000","last_fullsized":"2024-09-17T03:14:16.390671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_clean_scrub_stamp":"2024-09-17T03:14:03.847721+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:32.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333439+0000","last_change":"2024-09-17T03:14:13.318977+0000","last_active":"2024-09-17T03:14:15.333439+0000","last_peered":"2024-09-17T03:14:15.333439+0000","last_clean":"2024-09-17T03:14:15.333439+0000","last_became_active":"2024-09-17T03:14:13.318294+0000","last_became_peered":"2024-09-17T03:14:13.318294+0000","last_unstale":"2024-09-17T03:14:15.333439+0000","last_undegraded":"2024-09-17T03:14:15.333439+0000","last_fullsized":"2024-09-17T03:14:15.333439+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:39.634907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333413+0000","last_change":"2024-09-17T03:14:14.331745+0000","last_active":"2024-09-17T03:14:15.333413+0000","last_peered":"2024-09-17T03:14:15.333413+0000","last_clean":"2024-09-17T03:14:15.333413+0000","last_became_active":"2024-09-17T03:14:14.331207+0000","last_became_peered":"2024-09-17T03:14:14.331207+0000","last_unstale":"2024-09-17T03:14:15.333413+0000","last_undegraded":"2024-09-17T03:14:15.333413+0000","last_fullsized":"2024-09-17T03:14:15.333413+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:53:48.931242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332808+0000","last_change":"2024-09-17T03:14:13.324555+0000","last_active":"2024-09-17T03:14:15.332808+0000","last_peered":"2024-09-17T03:14:15.332808+0000","last_clean":"2024-09-17T03:14:15.332808+0000","last_became_active":"2024-09-17T03:14:13.324214+0000","last_became_peered":"2024-09-17T03:14:13.324214+0000","last_unstale":"2024-09-17T03:14:15.332808+0000","last_undegraded":"2024-09-17T03:14:15.332808+0000","last_fullsized":"2024-09-17T03:14:15.332808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:13.693714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391471+0000","last_change":"2024-09-17T03:14:14.339552+0000","last_active":"2024-09-17T03:14:16.391471+0000","last_peered":"2024-09-17T03:14:16.391471+0000","last_clean":"2024-09-17T03:14:16.391471+0000","last_became_active":"2024-09-17T03:14:14.339232+0000","last_became_peered":"2024-09-17T03:14:14.339232+0000","last_unstale":"2024-09-17T03:14:16.391471+0000","last_undegraded":"2024-09-17T03:14:16.391471+0000","last_fullsized":"2024-09-17T03:14:16.391471+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:31.035937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391696+0000","last_change":"2024-09-17T03:14:13.320582+0000","last_active":"2024-09-17T03:14:16.391696+0000","last_peered":"2024-09-17T03:14:16.391696+0000","last_clean":"2024-09-17T03:14:16.391696+0000","last_became_active":"2024-09-17T03:14:13.320217+0000","last_became_peered":"2024-09-17T03:14:13.320217+0000","last_unstale":"2024-09-17T03:14:16.391696+0000","last_undegraded":"2024-09-17T03:14:16.391696+0000","last_fullsized":"2024-09-17T03:14:16.391696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:34.405796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334392+0000","last_change":"2024-09-17T03:14:14.334667+0000","last_active":"2024-09-17T03:14:15.334392+0000","last_peered":"2024-09-17T03:14:15.334392+0000","last_clean":"2024-09-17T03:14:15.334392+0000","last_became_active":"2024-09-17T03:14:14.333112+0000","last_became_peered":"2024-09-17T03:14:14.333112+0000","last_unstale":"2024-09-17T03:14:15.334392+0000","last_undegraded":"2024-09-17T03:14:15.334392+0000","last_fullsized":"2024-09-17T03:14:15.334392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:17.642423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334025+0000","last_change":"2024-09-17T03:14:13.327887+0000","last_active":"2024-09-17T03:14:15.334025+0000","last_peered":"2024-09-17T03:14:15.334025+0000","last_clean":"2024-09-17T03:14:15.334025+0000","last_became_active":"2024-09-17T03:14:13.327514+0000","last_became_peered":"2024-09-17T03:14:13.327514+0000","last_unstale":"2024-09-17T03:14:15.334025+0000","last_undegraded":"2024-09-17T03:14:15.334025+0000","last_fullsized":"2024-09-17T03:14:15.334025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:10:41.615501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333005+0000","last_change":"2024-09-17T03:14:14.338404+0000","last_active":"2024-09-17T03:14:15.333005+0000","last_peered":"2024-09-17T03:14:15.333005+0000","last_clean":"2024-09-17T03:14:15.333005+0000","last_became_active":"2024-09-17T03:14:14.338120+0000","last_became_peered":"2024-09-17T03:14:14.338120+0000","last_unstale":"2024-09-17T03:14:15.333005+0000","last_undegraded":"2024-09-17T03:14:15.333005+0000","last_fullsized":"2024-09-17T03:14:15.333005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:17.576471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334665+0000","last_change":"2024-09-17T03:14:13.324534+0000","last_active":"2024-09-17T03:14:15.334665+0000","last_peered":"2024-09-17T03:14:15.334665+0000","last_clean":"2024-09-17T03:14:15.334665+0000","last_became_active":"2024-09-17T03:14:13.324173+0000","last_became_peered":"2024-09-17T03:14:13.324173+0000","last_unstale":"2024-09-17T03:14:15.334665+0000","last_undegraded":"2024-09-17T03:14:15.334665+0000","last_fullsized":"2024-09-17T03:14:15.334665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:20.601544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391770+0000","last_change":"2024-09-17T03:14:14.341535+0000","last_active":"2024-09-17T03:14:16.391770+0000","last_peered":"2024-09-17T03:14:16.391770+0000","last_clean":"2024-09-17T03:14:16.391770+0000","last_became_active":"2024-09-17T03:14:14.339526+0000","last_became_peered":"2024-09-17T03:14:14.339526+0000","last_unstale":"2024-09-17T03:14:16.391770+0000","last_undegraded":"2024-09-17T03:14:16.391770+0000","last_fullsized":"2024-09-17T03:14:16.391770+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:45.268231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334336+0000","last_change":"2024-09-17T03:14:13.321581+0000","last_active":"2024-09-17T03:14:15.334336+0000","last_peered":"2024-09-17T03:14:15.334336+0000","last_clean":"2024-09-17T03:14:15.334336+0000","last_became_active":"2024-09-17T03:14:13.321238+0000","last_became_peered":"2024-09-17T03:14:13.321238+0000","last_unstale":"2024-09-17T03:14:15.334336+0000","last_undegraded":"2024-09-17T03:14:15.334336+0000","last_fullsized":"2024-09-17T03:14:15.334336+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:57.200786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392107+0000","last_change":"2024-09-17T03:14:14.343574+0000","last_active":"2024-09-17T03:14:16.392107+0000","last_peered":"2024-09-17T03:14:16.392107+0000","last_clean":"2024-09-17T03:14:16.392107+0000","last_became_active":"2024-09-17T03:14:14.342586+0000","last_became_peered":"2024-09-17T03:14:14.342586+0000","last_unstale":"2024-09-17T03:14:16.392107+0000","last_undegraded":"2024-09-17T03:14:16.392107+0000","last_fullsized":"2024-09-17T03:14:16.392107+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:53:20.317357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334149+0000","last_change":"2024-09-17T03:14:13.321756+0000","last_active":"2024-09-17T03:14:15.334149+0000","last_peered":"2024-09-17T03:14:15.334149+0000","last_clean":"2024-09-17T03:14:15.334149+0000","last_became_active":"2024-09-17T03:14:13.321392+0000","last_became_peered":"2024-09-17T03:14:13.321392+0000","last_unstale":"2024-09-17T03:14:15.334149+0000","last_undegraded":"2024-09-17T03:14:15.334149+0000","last_fullsized":"2024-09-17T03:14:15.334149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:33.594511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333517+0000","last_change":"2024-09-17T03:14:14.341909+0000","last_active":"2024-09-17T03:14:15.333517+0000","last_peered":"2024-09-17T03:14:15.333517+0000","last_clean":"2024-09-17T03:14:15.333517+0000","last_became_active":"2024-09-17T03:14:14.341571+0000","last_became_peered":"2024-09-17T03:14:14.341571+0000","last_unstale":"2024-09-17T03:14:15.333517+0000","last_undegraded":"2024-09-17T03:14:15.333517+0000","last_fullsized":"2024-09-17T03:14:15.333517+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:41.776202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334199+0000","last_change":"2024-09-17T03:14:13.329358+0000","last_active":"2024-09-17T03:14:15.334199+0000","last_peered":"2024-09-17T03:14:15.334199+0000","last_clean":"2024-09-17T03:14:15.334199+0000","last_became_active":"2024-09-17T03:14:13.328836+0000","last_became_peered":"2024-09-17T03:14:13.328836+0000","last_unstale":"2024-09-17T03:14:15.334199+0000","last_undegraded":"2024-09-17T03:14:15.334199+0000","last_fullsized":"2024-09-17T03:14:15.334199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:44.263456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333819+0000","last_change":"2024-09-17T03:14:14.340502+0000","last_active":"2024-09-17T03:14:15.333819+0000","last_peered":"2024-09-17T03:14:15.333819+0000","last_clean":"2024-09-17T03:14:15.333819+0000","last_became_active":"2024-09-17T03:14:14.340215+0000","last_became_peered":"2024-09-17T03:14:14.340215+0000","last_unstale":"2024-09-17T03:14:15.333819+0000","last_undegraded":"2024-09-17T03:14:15.333819+0000","last_fullsized":"2024-09-17T03:14:15.333819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:45.283163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332057+0000","last_change":"2024-09-17T03:14:13.324045+0000","last_active":"2024-09-17T03:14:15.332057+0000","last_peered":"2024-09-17T03:14:15.332057+0000","last_clean":"2024-09-17T03:14:15.332057+0000","last_became_active":"2024-09-17T03:14:13.323025+0000","last_became_peered":"2024-09-17T03:14:13.323025+0000","last_unstale":"2024-09-17T03:14:15.332057+0000","last_undegraded":"2024-09-17T03:14:15.332057+0000","last_fullsized":"2024-09-17T03:14:15.332057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:02.157890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333680+0000","last_change":"2024-09-17T03:14:14.346858+0000","last_active":"2024-09-17T03:14:15.333680+0000","last_peered":"2024-09-17T03:14:15.333680+0000","last_clean":"2024-09-17T03:14:15.333680+0000","last_became_active":"2024-09-17T03:14:14.346576+0000","last_became_peered":"2024-09-17T03:14:14.346576+0000","last_unstale":"2024-09-17T03:14:15.333680+0000","last_undegraded":"2024-09-17T03:14:15.333680+0000","last_fullsized":"2024-09-17T03:14:15.333680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:38.080821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334132+0000","last_change":"2024-09-17T03:14:13.319558+0000","last_active":"2024-09-17T03:14:15.334132+0000","last_peered":"2024-09-17T03:14:15.334132+0000","last_clean":"2024-09-17T03:14:15.334132+0000","last_became_active":"2024-09-17T03:14:13.319236+0000","last_became_peered":"2024-09-17T03:14:13.319236+0000","last_unstale":"2024-09-17T03:14:15.334132+0000","last_undegraded":"2024-09-17T03:14:15.334132+0000","last_fullsized":"2024-09-17T03:14:15.334132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:25:39.260408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333163+0000","last_change":"2024-09-17T03:14:14.329694+0000","last_active":"2024-09-17T03:14:15.333163+0000","last_peered":"2024-09-17T03:14:15.333163+0000","last_clean":"2024-09-17T03:14:15.333163+0000","last_became_active":"2024-09-17T03:14:14.329289+0000","last_became_peered":"2024-09-17T03:14:14.329289+0000","last_unstale":"2024-09-17T03:14:15.333163+0000","last_undegraded":"2024-09-17T03:14:15.333163+0000","last_fullsized":"2024-09-17T03:14:15.333163+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:59.114330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392121+0000","last_change":"2024-09-17T03:14:13.323443+0000","last_active":"2024-09-17T03:14:16.392121+0000","last_peered":"2024-09-17T03:14:16.392121+0000","last_clean":"2024-09-17T03:14:16.392121+0000","last_became_active":"2024-09-17T03:14:13.323047+0000","last_became_peered":"2024-09-17T03:14:13.323047+0000","last_unstale":"2024-09-17T03:14:16.392121+0000","last_undegraded":"2024-09-17T03:14:16.392121+0000","last_fullsized":"2024-09-17T03:14:16.392121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:34.405796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392246+0000","last_change":"2024-09-17T03:14:14.336168+0000","last_active":"2024-09-17T03:14:16.392246+0000","last_peered":"2024-09-17T03:14:16.392246+0000","last_clean":"2024-09-17T03:14:16.392246+0000","last_became_active":"2024-09-17T03:14:14.335860+0000","last_became_peered":"2024-09-17T03:14:14.335860+0000","last_unstale":"2024-09-17T03:14:16.392246+0000","last_undegraded":"2024-09-17T03:14:16.392246+0000","last_fullsized":"2024-09-17T03:14:16.392246+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:40.435687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333326+0000","last_change":"2024-09-17T03:14:13.319349+0000","last_active":"2024-09-17T03:14:15.333326+0000","last_peered":"2024-09-17T03:14:15.333326+0000","last_clean":"2024-09-17T03:14:15.333326+0000","last_became_active":"2024-09-17T03:14:13.318677+0000","last_became_peered":"2024-09-17T03:14:13.318677+0000","last_unstale":"2024-09-17T03:14:15.333326+0000","last_undegraded":"2024-09-17T03:14:15.333326+0000","last_fullsized":"2024-09-17T03:14:15.333326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:55:19.372118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333793+0000","last_change":"2024-09-17T03:14:14.343651+0000","last_active":"2024-09-17T03:14:15.333793+0000","last_peered":"2024-09-17T03:14:15.333793+0000","last_clean":"2024-09-17T03:14:15.333793+0000","last_became_active":"2024-09-17T03:14:14.343219+0000","last_became_peered":"2024-09-17T03:14:14.343219+0000","last_unstale":"2024-09-17T03:14:15.333793+0000","last_undegraded":"2024-09-17T03:14:15.333793+0000","last_fullsized":"2024-09-17T03:14:15.333793+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:32.656876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333968+0000","last_change":"2024-09-17T03:14:14.347881+0000","last_active":"2024-09-17T03:14:15.333968+0000","last_peered":"2024-09-17T03:14:15.333968+0000","last_clean":"2024-09-17T03:14:15.333968+0000","last_became_active":"2024-09-17T03:14:14.346360+0000","last_became_peered":"2024-09-17T03:14:14.346360+0000","last_unstale":"2024-09-17T03:14:15.333968+0000","last_undegraded":"2024-09-17T03:14:15.333968+0000","last_fullsized":"2024-09-17T03:14:15.333968+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:21.993789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333484+0000","last_change":"2024-09-17T03:14:13.329489+0000","last_active":"2024-09-17T03:14:15.333484+0000","last_peered":"2024-09-17T03:14:15.333484+0000","last_clean":"2024-09-17T03:14:15.333484+0000","last_became_active":"2024-09-17T03:14:13.329141+0000","last_became_peered":"2024-09-17T03:14:13.329141+0000","last_unstale":"2024-09-17T03:14:15.333484+0000","last_undegraded":"2024-09-17T03:14:15.333484+0000","last_fullsized":"2024-09-17T03:14:15.333484+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:44.263456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391814+0000","last_change":"2024-09-17T03:14:14.337492+0000","last_active":"2024-09-17T03:14:16.391814+0000","last_peered":"2024-09-17T03:14:16.391814+0000","last_clean":"2024-09-17T03:14:16.391814+0000","last_became_active":"2024-09-17T03:14:14.337160+0000","last_became_peered":"2024-09-17T03:14:14.337160+0000","last_unstale":"2024-09-17T03:14:16.391814+0000","last_undegraded":"2024-09-17T03:14:16.391814+0000","last_fullsized":"2024-09-17T03:14:16.391814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:31.035937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333395+0000","last_change":"2024-09-17T03:14:13.323396+0000","last_active":"2024-09-17T03:14:15.333395+0000","last_peered":"2024-09-17T03:14:15.333395+0000","last_clean":"2024-09-17T03:14:15.333395+0000","last_became_active":"2024-09-17T03:14:13.322911+0000","last_became_peered":"2024-09-17T03:14:13.322911+0000","last_unstale":"2024-09-17T03:14:15.333395+0000","last_undegraded":"2024-09-17T03:14:15.333395+0000","last_fullsized":"2024-09-17T03:14:15.333395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:36.650878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.335094+0000","last_change":"2024-09-17T03:14:14.340359+0000","last_active":"2024-09-17T03:14:15.335094+0000","last_peered":"2024-09-17T03:14:15.335094+0000","last_clean":"2024-09-17T03:14:15.335094+0000","last_became_active":"2024-09-17T03:14:14.340060+0000","last_became_peered":"2024-09-17T03:14:14.340060+0000","last_unstale":"2024-09-17T03:14:15.335094+0000","last_undegraded":"2024-09-17T03:14:15.335094+0000","last_fullsized":"2024-09-17T03:14:15.335094+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:59:36.040787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334845+0000","last_change":"2024-09-17T03:14:13.317875+0000","last_active":"2024-09-17T03:14:15.334845+0000","last_peered":"2024-09-17T03:14:15.334845+0000","last_clean":"2024-09-17T03:14:15.334845+0000","last_became_active":"2024-09-17T03:14:13.317521+0000","last_became_peered":"2024-09-17T03:14:13.317521+0000","last_unstale":"2024-09-17T03:14:15.334845+0000","last_undegraded":"2024-09-17T03:14:15.334845+0000","last_fullsized":"2024-09-17T03:14:15.334845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:05.715850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.335048+0000","last_change":"2024-09-17T03:14:14.338400+0000","last_active":"2024-09-17T03:14:15.335048+0000","last_peered":"2024-09-17T03:14:15.335048+0000","last_clean":"2024-09-17T03:14:15.335048+0000","last_became_active":"2024-09-17T03:14:14.338003+0000","last_became_peered":"2024-09-17T03:14:14.338003+0000","last_unstale":"2024-09-17T03:14:15.335048+0000","last_undegraded":"2024-09-17T03:14:15.335048+0000","last_fullsized":"2024-09-17T03:14:15.335048+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:57:32.024812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333702+0000","last_change":"2024-09-17T03:14:13.324341+0000","last_active":"2024-09-17T03:14:15.333702+0000","last_peered":"2024-09-17T03:14:15.333702+0000","last_clean":"2024-09-17T03:14:15.333702+0000","last_became_active":"2024-09-17T03:14:13.323959+0000","last_became_peered":"2024-09-17T03:14:13.323959+0000","last_unstale":"2024-09-17T03:14:15.333702+0000","last_undegraded":"2024-09-17T03:14:15.333702+0000","last_fullsized":"2024-09-17T03:14:15.333702+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:18:06.456762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":82231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":82107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27844,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343996,"statfs":{"total":96636764160,"available":96608251904,"internally_reserved":0,"allocated":835584,"data_stored":672567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":82689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":245760,"data_stored":83719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":672545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":82163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":84391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:14:26.649 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:14:26.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.800+0000 7f8a48415640 1 -- 172.21.15.87:0/3851332214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a2c004800 msgr2=0x7f8a2c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.800+0000 7f8a48415640 1 --2- 172.21.15.87:0/3851332214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a2c004800 0x7f8a2c004c10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8a300099d0 tx=0x7f8a2c005830 comp rx=0 tx=0).stop 2024-09-17T03:14:26.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.800+0000 7f8a469b2640 1 -- 172.21.15.87:0/3851332214 reap_dead start 2024-09-17T03:14:26.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.800+0000 7f8a48415640 1 -- 172.21.15.87:0/3851332214 shutdown_connections 2024-09-17T03:14:26.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.800+0000 7f8a48415640 1 -- 172.21.15.87:0/3851332214 >> 172.21.15.87:0/3851332214 conn(0x7f8a40069900 msgr2=0x7f8a400d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:26.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.800+0000 7f8a48415640 1 -- 172.21.15.87:0/3851332214 shutdown_connections 2024-09-17T03:14:26.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.800+0000 7f8a48415640 1 -- 172.21.15.87:0/3851332214 wait complete. 2024-09-17T03:14:26.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.801+0000 7f8a48415640 1 Processor -- start 2024-09-17T03:14:26.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.801+0000 7f8a48415640 1 -- start start 2024-09-17T03:14:26.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.803+0000 7f8a48415640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a2c004800 0x7f8a40149650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:26.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.803+0000 7f8a48415640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a401f9460 0x7f8a400cd630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:26.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.803+0000 7f8a48415640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a400cdb70 0x7f8a4014a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:26.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.803+0000 7f8a48415640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8a400b67f0 con 0x7f8a401f9460 2024-09-17T03:14:26.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.803+0000 7f8a48415640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8a400b6670 con 0x7f8a2c004800 2024-09-17T03:14:26.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.803+0000 7f8a48415640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8a400d3720 con 0x7f8a400cdb70 2024-09-17T03:14:26.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.803+0000 7f8a459b0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a401f9460 0x7f8a400cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:26.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.803+0000 7f8a459b0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a401f9460 0x7f8a400cd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50880/0 (socket says 172.21.15.87:50880) 2024-09-17T03:14:26.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.803+0000 7f8a459b0640 1 -- 172.21.15.87:0/319617955 learned_addr learned my addr 172.21.15.87:0/319617955 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:26.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.803+0000 7f8a469b2640 1 --2- 172.21.15.87:0/319617955 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a400cdb70 0x7f8a4014a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:26.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.803+0000 7f8a461b1640 1 --2- 172.21.15.87:0/319617955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a2c004800 0x7f8a40149650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:26.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.803+0000 7f8a459b0640 1 -- 172.21.15.87:0/319617955 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a400cdb70 msgr2=0x7f8a4014a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.803+0000 7f8a459b0640 1 --2- 172.21.15.87:0/319617955 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a400cdb70 0x7f8a4014a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.803+0000 7f8a459b0640 1 -- 172.21.15.87:0/319617955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a2c004800 msgr2=0x7f8a40149650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.803+0000 7f8a459b0640 1 --2- 172.21.15.87:0/319617955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a2c004800 0x7f8a40149650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.803+0000 7f8a459b0640 1 -- 172.21.15.87:0/319617955 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a400cdf80 con 0x7f8a401f9460 2024-09-17T03:14:26.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.803+0000 7f8a459b0640 1 --2- 172.21.15.87:0/319617955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a401f9460 0x7f8a400cd630 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8a300098f0 tx=0x7f8a30003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:26.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.803+0000 7f8a36ffd640 1 -- 172.21.15.87:0/319617955 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30037050 con 0x7f8a401f9460 2024-09-17T03:14:26.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.803+0000 7f8a36ffd640 1 -- 172.21.15.87:0/319617955 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a30043070 con 0x7f8a401f9460 2024-09-17T03:14:26.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.803+0000 7f8a48415640 1 -- 172.21.15.87:0/319617955 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a401f9ee0 con 0x7f8a401f9460 2024-09-17T03:14:26.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.803+0000 7f8a36ffd640 1 -- 172.21.15.87:0/319617955 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30005be0 con 0x7f8a401f9460 2024-09-17T03:14:26.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.803+0000 7f8a48415640 1 -- 172.21.15.87:0/319617955 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a4014a060 con 0x7f8a401f9460 2024-09-17T03:14:26.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.804+0000 7f8a48415640 1 -- 172.21.15.87:0/319617955 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a401f9c50 con 0x7f8a401f9460 2024-09-17T03:14:26.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.805+0000 7f8a36ffd640 1 -- 172.21.15.87:0/319617955 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8a30032850 con 0x7f8a401f9460 2024-09-17T03:14:26.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.805+0000 7f8a36ffd640 1 --2- 172.21.15.87:0/319617955 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8a24078bc0 0x7f8a2407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:26.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.806+0000 7f8a36ffd640 1 -- 172.21.15.87:0/319617955 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f8a300bba90 con 0x7f8a401f9460 2024-09-17T03:14:26.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.806+0000 7f8a461b1640 1 --2- 172.21.15.87:0/319617955 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8a24078bc0 0x7f8a2407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:26.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.806+0000 7f8a461b1640 1 --2- 172.21.15.87:0/319617955 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8a24078bc0 0x7f8a2407b0d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c004700 tx=0x7f8a3c005e30 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:26.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.810+0000 7f8a36ffd640 1 -- 172.21.15.87:0/319617955 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8a400cdf80 con 0x7f8a401f9460 2024-09-17T03:14:26.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.980+0000 7f8a48415640 1 -- 172.21.15.87:0/319617955 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8a400b5ea0 con 0x7f8a24078bc0 2024-09-17T03:14:26.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.985+0000 7f8a36ffd640 1 -- 172.21.15.87:0/319617955 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251821 (secure 0 0 0) 0x7f8a400b5ea0 con 0x7f8a24078bc0 2024-09-17T03:14:26.986 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:26.987 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-09-17T03:14:26.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.988+0000 7f8a48415640 1 -- 172.21.15.87:0/319617955 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8a24078bc0 msgr2=0x7f8a2407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.988+0000 7f8a48415640 1 --2- 172.21.15.87:0/319617955 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8a24078bc0 0x7f8a2407b0d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c004700 tx=0x7f8a3c005e30 comp rx=0 tx=0).stop 2024-09-17T03:14:26.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.988+0000 7f8a48415640 1 -- 172.21.15.87:0/319617955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a401f9460 msgr2=0x7f8a400cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:26.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.988+0000 7f8a48415640 1 --2- 172.21.15.87:0/319617955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a401f9460 0x7f8a400cd630 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8a300098f0 tx=0x7f8a30003820 comp rx=0 tx=0).stop 2024-09-17T03:14:26.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.988+0000 7f8a48415640 1 -- 172.21.15.87:0/319617955 shutdown_connections 2024-09-17T03:14:26.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.988+0000 7f8a48415640 1 --2- 172.21.15.87:0/319617955 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8a24078bc0 0x7f8a2407b0d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.988+0000 7f8a48415640 1 --2- 172.21.15.87:0/319617955 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8a400cdb70 0x7f8a4014a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.988+0000 7f8a48415640 1 --2- 172.21.15.87:0/319617955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a401f9460 0x7f8a400cd630 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.988+0000 7f8a48415640 1 --2- 172.21.15.87:0/319617955 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8a2c004800 0x7f8a40149650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:26.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.988+0000 7f8a48415640 1 -- 172.21.15.87:0/319617955 >> 172.21.15.87:0/319617955 conn(0x7f8a40069900 msgr2=0x7f8a4006de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:26.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.988+0000 7f8a48415640 1 -- 172.21.15.87:0/319617955 shutdown_connections 2024-09-17T03:14:26.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:26.988+0000 7f8a48415640 1 -- 172.21.15.87:0/319617955 wait complete. 2024-09-17T03:14:26.998 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-09-17T03:14:25.419310+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219280,"kb_used_data":3048,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755440,"statfs":{"total":773094113280,"available":772869570560,"internally_reserved":0,"allocated":3121152,"data_stored":1842412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.087903"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332877+0000","last_change":"2024-09-17T03:14:14.350186+0000","last_active":"2024-09-17T03:14:15.332877+0000","last_peered":"2024-09-17T03:14:15.332877+0000","last_clean":"2024-09-17T03:14:15.332877+0000","last_became_active":"2024-09-17T03:14:14.349855+0000","last_became_peered":"2024-09-17T03:14:14.349855+0000","last_unstale":"2024-09-17T03:14:15.332877+0000","last_undegraded":"2024-09-17T03:14:15.332877+0000","last_fullsized":"2024-09-17T03:14:15.332877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:56:09.730649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334764+0000","last_change":"2024-09-17T03:14:14.343284+0000","last_active":"2024-09-17T03:14:15.334764+0000","last_peered":"2024-09-17T03:14:15.334764+0000","last_clean":"2024-09-17T03:14:15.334764+0000","last_became_active":"2024-09-17T03:14:14.337583+0000","last_became_peered":"2024-09-17T03:14:14.337583+0000","last_unstale":"2024-09-17T03:14:15.334764+0000","last_undegraded":"2024-09-17T03:14:15.334764+0000","last_fullsized":"2024-09-17T03:14:15.334764+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:06:49.022656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334676+0000","last_change":"2024-09-17T03:14:14.340288+0000","last_active":"2024-09-17T03:14:15.334676+0000","last_peered":"2024-09-17T03:14:15.334676+0000","last_clean":"2024-09-17T03:14:15.334676+0000","last_became_active":"2024-09-17T03:14:14.338799+0000","last_became_peered":"2024-09-17T03:14:14.338799+0000","last_unstale":"2024-09-17T03:14:15.334676+0000","last_undegraded":"2024-09-17T03:14:15.334676+0000","last_fullsized":"2024-09-17T03:14:15.334676+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:16:54.254992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391881+0000","last_change":"2024-09-17T03:14:14.333722+0000","last_active":"2024-09-17T03:14:16.391881+0000","last_peered":"2024-09-17T03:14:16.391881+0000","last_clean":"2024-09-17T03:14:16.391881+0000","last_became_active":"2024-09-17T03:14:14.333401+0000","last_became_peered":"2024-09-17T03:14:14.333401+0000","last_unstale":"2024-09-17T03:14:16.391881+0000","last_undegraded":"2024-09-17T03:14:16.391881+0000","last_fullsized":"2024-09-17T03:14:16.391881+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:40.499497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333292+0000","last_change":"2024-09-17T03:14:14.334841+0000","last_active":"2024-09-17T03:14:15.333292+0000","last_peered":"2024-09-17T03:14:15.333292+0000","last_clean":"2024-09-17T03:14:15.333292+0000","last_became_active":"2024-09-17T03:14:14.334543+0000","last_became_peered":"2024-09-17T03:14:14.334543+0000","last_unstale":"2024-09-17T03:14:15.333292+0000","last_undegraded":"2024-09-17T03:14:15.333292+0000","last_fullsized":"2024-09-17T03:14:15.333292+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:50:51.158396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391311+0000","last_change":"2024-09-17T03:14:14.339302+0000","last_active":"2024-09-17T03:14:16.391311+0000","last_peered":"2024-09-17T03:14:16.391311+0000","last_clean":"2024-09-17T03:14:16.391311+0000","last_became_active":"2024-09-17T03:14:14.331863+0000","last_became_peered":"2024-09-17T03:14:14.331863+0000","last_unstale":"2024-09-17T03:14:16.391311+0000","last_undegraded":"2024-09-17T03:14:16.391311+0000","last_fullsized":"2024-09-17T03:14:16.391311+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:40.639682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332440+0000","last_change":"2024-09-17T03:14:14.340253+0000","last_active":"2024-09-17T03:14:15.332440+0000","last_peered":"2024-09-17T03:14:15.332440+0000","last_clean":"2024-09-17T03:14:15.332440+0000","last_became_active":"2024-09-17T03:14:14.339855+0000","last_became_peered":"2024-09-17T03:14:14.339855+0000","last_unstale":"2024-09-17T03:14:15.332440+0000","last_undegraded":"2024-09-17T03:14:15.332440+0000","last_fullsized":"2024-09-17T03:14:15.332440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:44:20.684218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334380+0000","last_change":"2024-09-17T03:14:14.333479+0000","last_active":"2024-09-17T03:14:15.334380+0000","last_peered":"2024-09-17T03:14:15.334380+0000","last_clean":"2024-09-17T03:14:15.334380+0000","last_became_active":"2024-09-17T03:14:14.330382+0000","last_became_peered":"2024-09-17T03:14:14.330382+0000","last_unstale":"2024-09-17T03:14:15.334380+0000","last_undegraded":"2024-09-17T03:14:15.334380+0000","last_fullsized":"2024-09-17T03:14:15.334380+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:28:21.969469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391159+0000","last_change":"2024-09-17T03:14:14.345634+0000","last_active":"2024-09-17T03:14:16.391159+0000","last_peered":"2024-09-17T03:14:16.391159+0000","last_clean":"2024-09-17T03:14:16.391159+0000","last_became_active":"2024-09-17T03:14:14.345309+0000","last_became_peered":"2024-09-17T03:14:14.345309+0000","last_unstale":"2024-09-17T03:14:16.391159+0000","last_undegraded":"2024-09-17T03:14:16.391159+0000","last_fullsized":"2024-09-17T03:14:16.391159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:09.963317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332555+0000","last_change":"2024-09-17T03:14:14.337246+0000","last_active":"2024-09-17T03:14:15.332555+0000","last_peered":"2024-09-17T03:14:15.332555+0000","last_clean":"2024-09-17T03:14:15.332555+0000","last_became_active":"2024-09-17T03:14:14.336920+0000","last_became_peered":"2024-09-17T03:14:14.336920+0000","last_unstale":"2024-09-17T03:14:15.332555+0000","last_undegraded":"2024-09-17T03:14:15.332555+0000","last_fullsized":"2024-09-17T03:14:15.332555+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:35.974141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332769+0000","last_change":"2024-09-17T03:14:14.349173+0000","last_active":"2024-09-17T03:14:15.332769+0000","last_peered":"2024-09-17T03:14:15.332769+0000","last_clean":"2024-09-17T03:14:15.332769+0000","last_became_active":"2024-09-17T03:14:14.348420+0000","last_became_peered":"2024-09-17T03:14:14.348420+0000","last_unstale":"2024-09-17T03:14:15.332769+0000","last_undegraded":"2024-09-17T03:14:15.332769+0000","last_fullsized":"2024-09-17T03:14:15.332769+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:09.339640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332966+0000","last_change":"2024-09-17T03:14:14.336719+0000","last_active":"2024-09-17T03:14:15.332966+0000","last_peered":"2024-09-17T03:14:15.332966+0000","last_clean":"2024-09-17T03:14:15.332966+0000","last_became_active":"2024-09-17T03:14:14.336239+0000","last_became_peered":"2024-09-17T03:14:14.336239+0000","last_unstale":"2024-09-17T03:14:15.332966+0000","last_undegraded":"2024-09-17T03:14:15.332966+0000","last_fullsized":"2024-09-17T03:14:15.332966+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:15:59.839348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391861+0000","last_change":"2024-09-17T03:14:14.340337+0000","last_active":"2024-09-17T03:14:16.391861+0000","last_peered":"2024-09-17T03:14:16.391861+0000","last_clean":"2024-09-17T03:14:16.391861+0000","last_became_active":"2024-09-17T03:14:14.340028+0000","last_became_peered":"2024-09-17T03:14:14.340028+0000","last_unstale":"2024-09-17T03:14:16.391861+0000","last_undegraded":"2024-09-17T03:14:16.391861+0000","last_fullsized":"2024-09-17T03:14:16.391861+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:54:35.806642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392067+0000","last_change":"2024-09-17T03:14:14.334276+0000","last_active":"2024-09-17T03:14:16.392067+0000","last_peered":"2024-09-17T03:14:16.392067+0000","last_clean":"2024-09-17T03:14:16.392067+0000","last_became_active":"2024-09-17T03:14:14.333764+0000","last_became_peered":"2024-09-17T03:14:14.333764+0000","last_unstale":"2024-09-17T03:14:16.392067+0000","last_undegraded":"2024-09-17T03:14:16.392067+0000","last_fullsized":"2024-09-17T03:14:16.392067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:57.801941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333421+0000","last_change":"2024-09-17T03:14:14.337055+0000","last_active":"2024-09-17T03:14:15.333421+0000","last_peered":"2024-09-17T03:14:15.333421+0000","last_clean":"2024-09-17T03:14:15.333421+0000","last_became_active":"2024-09-17T03:14:14.334416+0000","last_became_peered":"2024-09-17T03:14:14.334416+0000","last_unstale":"2024-09-17T03:14:15.333421+0000","last_undegraded":"2024-09-17T03:14:15.333421+0000","last_fullsized":"2024-09-17T03:14:15.333421+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:12:03.846825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332440+0000","last_change":"2024-09-17T03:14:14.346619+0000","last_active":"2024-09-17T03:14:15.332440+0000","last_peered":"2024-09-17T03:14:15.332440+0000","last_clean":"2024-09-17T03:14:15.332440+0000","last_became_active":"2024-09-17T03:14:14.346322+0000","last_became_peered":"2024-09-17T03:14:14.346322+0000","last_unstale":"2024-09-17T03:14:15.332440+0000","last_undegraded":"2024-09-17T03:14:15.332440+0000","last_fullsized":"2024-09-17T03:14:15.332440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:34:00.384055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334241+0000","last_change":"2024-09-17T03:14:14.343978+0000","last_active":"2024-09-17T03:14:15.334241+0000","last_peered":"2024-09-17T03:14:15.334241+0000","last_clean":"2024-09-17T03:14:15.334241+0000","last_became_active":"2024-09-17T03:14:14.343600+0000","last_became_peered":"2024-09-17T03:14:14.343600+0000","last_unstale":"2024-09-17T03:14:15.334241+0000","last_undegraded":"2024-09-17T03:14:15.334241+0000","last_fullsized":"2024-09-17T03:14:15.334241+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:36:25.368172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391936+0000","last_change":"2024-09-17T03:14:14.336744+0000","last_active":"2024-09-17T03:14:16.391936+0000","last_peered":"2024-09-17T03:14:16.391936+0000","last_clean":"2024-09-17T03:14:16.391936+0000","last_became_active":"2024-09-17T03:14:14.336428+0000","last_became_peered":"2024-09-17T03:14:14.336428+0000","last_unstale":"2024-09-17T03:14:16.391936+0000","last_undegraded":"2024-09-17T03:14:16.391936+0000","last_fullsized":"2024-09-17T03:14:16.391936+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:32:09.963317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334349+0000","last_change":"2024-09-17T03:14:14.348250+0000","last_active":"2024-09-17T03:14:15.334349+0000","last_peered":"2024-09-17T03:14:15.334349+0000","last_clean":"2024-09-17T03:14:15.334349+0000","last_became_active":"2024-09-17T03:14:14.347538+0000","last_became_peered":"2024-09-17T03:14:14.347538+0000","last_unstale":"2024-09-17T03:14:15.334349+0000","last_undegraded":"2024-09-17T03:14:15.334349+0000","last_fullsized":"2024-09-17T03:14:15.334349+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:09:46.734553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333193+0000","last_change":"2024-09-17T03:14:14.335802+0000","last_active":"2024-09-17T03:14:15.333193+0000","last_peered":"2024-09-17T03:14:15.333193+0000","last_clean":"2024-09-17T03:14:15.333193+0000","last_became_active":"2024-09-17T03:14:14.335507+0000","last_became_peered":"2024-09-17T03:14:14.335507+0000","last_unstale":"2024-09-17T03:14:15.333193+0000","last_undegraded":"2024-09-17T03:14:15.333193+0000","last_fullsized":"2024-09-17T03:14:15.333193+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:30:29.705538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333725+0000","last_change":"2024-09-17T03:14:14.338758+0000","last_active":"2024-09-17T03:14:15.333725+0000","last_peered":"2024-09-17T03:14:15.333725+0000","last_clean":"2024-09-17T03:14:15.333725+0000","last_became_active":"2024-09-17T03:14:14.337821+0000","last_became_peered":"2024-09-17T03:14:14.337821+0000","last_unstale":"2024-09-17T03:14:15.333725+0000","last_undegraded":"2024-09-17T03:14:15.333725+0000","last_fullsized":"2024-09-17T03:14:15.333725+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:09:24.347980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391355+0000","last_change":"2024-09-17T03:14:14.332615+0000","last_active":"2024-09-17T03:14:16.391355+0000","last_peered":"2024-09-17T03:14:16.391355+0000","last_clean":"2024-09-17T03:14:16.391355+0000","last_became_active":"2024-09-17T03:14:14.332285+0000","last_became_peered":"2024-09-17T03:14:14.332285+0000","last_unstale":"2024-09-17T03:14:16.391355+0000","last_undegraded":"2024-09-17T03:14:16.391355+0000","last_fullsized":"2024-09-17T03:14:16.391355+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:40.435687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333687+0000","last_change":"2024-09-17T03:14:14.342718+0000","last_active":"2024-09-17T03:14:15.333687+0000","last_peered":"2024-09-17T03:14:15.333687+0000","last_clean":"2024-09-17T03:14:15.333687+0000","last_became_active":"2024-09-17T03:14:14.341029+0000","last_became_peered":"2024-09-17T03:14:14.341029+0000","last_unstale":"2024-09-17T03:14:15.333687+0000","last_undegraded":"2024-09-17T03:14:15.333687+0000","last_fullsized":"2024-09-17T03:14:15.333687+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:24:30.789181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332649+0000","last_change":"2024-09-17T03:14:14.344909+0000","last_active":"2024-09-17T03:14:15.332649+0000","last_peered":"2024-09-17T03:14:15.332649+0000","last_clean":"2024-09-17T03:14:15.332649+0000","last_became_active":"2024-09-17T03:14:14.343198+0000","last_became_peered":"2024-09-17T03:14:14.343198+0000","last_unstale":"2024-09-17T03:14:15.332649+0000","last_undegraded":"2024-09-17T03:14:15.332649+0000","last_fullsized":"2024-09-17T03:14:15.332649+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:06.251650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334992+0000","last_change":"2024-09-17T03:14:14.339599+0000","last_active":"2024-09-17T03:14:15.334992+0000","last_peered":"2024-09-17T03:14:15.334992+0000","last_clean":"2024-09-17T03:14:15.334992+0000","last_became_active":"2024-09-17T03:14:14.339248+0000","last_became_peered":"2024-09-17T03:14:14.339248+0000","last_unstale":"2024-09-17T03:14:15.334992+0000","last_undegraded":"2024-09-17T03:14:15.334992+0000","last_fullsized":"2024-09-17T03:14:15.334992+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:57:00.810832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392086+0000","last_change":"2024-09-17T03:14:14.347769+0000","last_active":"2024-09-17T03:14:16.392086+0000","last_peered":"2024-09-17T03:14:16.392086+0000","last_clean":"2024-09-17T03:14:16.392086+0000","last_became_active":"2024-09-17T03:14:14.347449+0000","last_became_peered":"2024-09-17T03:14:14.347449+0000","last_unstale":"2024-09-17T03:14:16.392086+0000","last_undegraded":"2024-09-17T03:14:16.392086+0000","last_fullsized":"2024-09-17T03:14:16.392086+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:14.698488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392056+0000","last_change":"2024-09-17T03:14:14.334291+0000","last_active":"2024-09-17T03:14:16.392056+0000","last_peered":"2024-09-17T03:14:16.392056+0000","last_clean":"2024-09-17T03:14:16.392056+0000","last_became_active":"2024-09-17T03:14:14.333993+0000","last_became_peered":"2024-09-17T03:14:14.333993+0000","last_unstale":"2024-09-17T03:14:16.392056+0000","last_undegraded":"2024-09-17T03:14:16.392056+0000","last_fullsized":"2024-09-17T03:14:16.392056+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:57.558924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392260+0000","last_change":"2024-09-17T03:14:14.344498+0000","last_active":"2024-09-17T03:14:16.392260+0000","last_peered":"2024-09-17T03:14:16.392260+0000","last_clean":"2024-09-17T03:14:16.392260+0000","last_became_active":"2024-09-17T03:14:14.344151+0000","last_became_peered":"2024-09-17T03:14:14.344151+0000","last_unstale":"2024-09-17T03:14:16.392260+0000","last_undegraded":"2024-09-17T03:14:16.392260+0000","last_fullsized":"2024-09-17T03:14:16.392260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:04:24.782583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334896+0000","last_change":"2024-09-17T03:14:14.327110+0000","last_active":"2024-09-17T03:14:15.334896+0000","last_peered":"2024-09-17T03:14:15.334896+0000","last_clean":"2024-09-17T03:14:15.334896+0000","last_became_active":"2024-09-17T03:14:14.326521+0000","last_became_peered":"2024-09-17T03:14:14.326521+0000","last_unstale":"2024-09-17T03:14:15.334896+0000","last_undegraded":"2024-09-17T03:14:15.334896+0000","last_fullsized":"2024-09-17T03:14:15.334896+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:27:42.804342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333824+0000","last_change":"2024-09-17T03:14:14.344297+0000","last_active":"2024-09-17T03:14:15.333824+0000","last_peered":"2024-09-17T03:14:15.333824+0000","last_clean":"2024-09-17T03:14:15.333824+0000","last_became_active":"2024-09-17T03:14:14.343941+0000","last_became_peered":"2024-09-17T03:14:14.343941+0000","last_unstale":"2024-09-17T03:14:15.333824+0000","last_undegraded":"2024-09-17T03:14:15.333824+0000","last_fullsized":"2024-09-17T03:14:15.333824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:18:47.095575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391581+0000","last_change":"2024-09-17T03:14:14.335330+0000","last_active":"2024-09-17T03:14:16.391581+0000","last_peered":"2024-09-17T03:14:16.391581+0000","last_clean":"2024-09-17T03:14:16.391581+0000","last_became_active":"2024-09-17T03:14:14.334989+0000","last_became_peered":"2024-09-17T03:14:14.334989+0000","last_unstale":"2024-09-17T03:14:16.391581+0000","last_undegraded":"2024-09-17T03:14:16.391581+0000","last_fullsized":"2024-09-17T03:14:16.391581+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:30.375807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391256+0000","last_change":"2024-09-17T03:14:14.334915+0000","last_active":"2024-09-17T03:14:16.391256+0000","last_peered":"2024-09-17T03:14:16.391256+0000","last_clean":"2024-09-17T03:14:16.391256+0000","last_became_active":"2024-09-17T03:14:14.334587+0000","last_became_peered":"2024-09-17T03:14:14.334587+0000","last_unstale":"2024-09-17T03:14:16.391256+0000","last_undegraded":"2024-09-17T03:14:16.391256+0000","last_fullsized":"2024-09-17T03:14:16.391256+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:55:20.376892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391571+0000","last_change":"2024-09-17T03:14:14.335030+0000","last_active":"2024-09-17T03:14:16.391571+0000","last_peered":"2024-09-17T03:14:16.391571+0000","last_clean":"2024-09-17T03:14:16.391571+0000","last_became_active":"2024-09-17T03:14:14.334722+0000","last_became_peered":"2024-09-17T03:14:14.334722+0000","last_unstale":"2024-09-17T03:14:16.391571+0000","last_undegraded":"2024-09-17T03:14:16.391571+0000","last_fullsized":"2024-09-17T03:14:16.391571+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:59:38.941073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334406+0000","last_change":"2024-09-17T03:14:13.319793+0000","last_active":"2024-09-17T03:14:15.334406+0000","last_peered":"2024-09-17T03:14:15.334406+0000","last_clean":"2024-09-17T03:14:15.334406+0000","last_became_active":"2024-09-17T03:14:13.319467+0000","last_became_peered":"2024-09-17T03:14:13.319467+0000","last_unstale":"2024-09-17T03:14:15.334406+0000","last_undegraded":"2024-09-17T03:14:15.334406+0000","last_fullsized":"2024-09-17T03:14:15.334406+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:06:54.724381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333960+0000","last_change":"2024-09-17T03:14:14.330562+0000","last_active":"2024-09-17T03:14:15.333960+0000","last_peered":"2024-09-17T03:14:15.333960+0000","last_clean":"2024-09-17T03:14:15.333960+0000","last_became_active":"2024-09-17T03:14:14.330255+0000","last_became_peered":"2024-09-17T03:14:14.330255+0000","last_unstale":"2024-09-17T03:14:15.333960+0000","last_undegraded":"2024-09-17T03:14:15.333960+0000","last_fullsized":"2024-09-17T03:14:15.333960+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:52:15.666625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333906+0000","last_change":"2024-09-17T03:14:13.324436+0000","last_active":"2024-09-17T03:14:15.333906+0000","last_peered":"2024-09-17T03:14:15.333906+0000","last_clean":"2024-09-17T03:14:15.333906+0000","last_became_active":"2024-09-17T03:14:13.323350+0000","last_became_peered":"2024-09-17T03:14:13.323350+0000","last_unstale":"2024-09-17T03:14:15.333906+0000","last_undegraded":"2024-09-17T03:14:15.333906+0000","last_fullsized":"2024-09-17T03:14:15.333906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:13.645071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391387+0000","last_change":"2024-09-17T03:14:13.330532+0000","last_active":"2024-09-17T03:14:16.391387+0000","last_peered":"2024-09-17T03:14:16.391387+0000","last_clean":"2024-09-17T03:14:16.391387+0000","last_became_active":"2024-09-17T03:14:13.330217+0000","last_became_peered":"2024-09-17T03:14:13.330217+0000","last_unstale":"2024-09-17T03:14:16.391387+0000","last_undegraded":"2024-09-17T03:14:16.391387+0000","last_fullsized":"2024-09-17T03:14:16.391387+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:00.266996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334660+0000","last_change":"2024-09-17T03:14:14.337981+0000","last_active":"2024-09-17T03:14:15.334660+0000","last_peered":"2024-09-17T03:14:15.334660+0000","last_clean":"2024-09-17T03:14:15.334660+0000","last_became_active":"2024-09-17T03:14:14.336212+0000","last_became_peered":"2024-09-17T03:14:14.336212+0000","last_unstale":"2024-09-17T03:14:15.334660+0000","last_undegraded":"2024-09-17T03:14:15.334660+0000","last_fullsized":"2024-09-17T03:14:15.334660+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:12:16.212384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332704+0000","last_change":"2024-09-17T03:14:13.316453+0000","last_active":"2024-09-17T03:14:15.332704+0000","last_peered":"2024-09-17T03:14:15.332704+0000","last_clean":"2024-09-17T03:14:15.332704+0000","last_became_active":"2024-09-17T03:14:13.316149+0000","last_became_peered":"2024-09-17T03:14:13.316149+0000","last_unstale":"2024-09-17T03:14:15.332704+0000","last_undegraded":"2024-09-17T03:14:15.332704+0000","last_fullsized":"2024-09-17T03:14:15.332704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:28:39.494722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334061+0000","last_change":"2024-09-17T03:14:14.339215+0000","last_active":"2024-09-17T03:14:15.334061+0000","last_peered":"2024-09-17T03:14:15.334061+0000","last_clean":"2024-09-17T03:14:15.334061+0000","last_became_active":"2024-09-17T03:14:14.337738+0000","last_became_peered":"2024-09-17T03:14:14.337738+0000","last_unstale":"2024-09-17T03:14:15.334061+0000","last_undegraded":"2024-09-17T03:14:15.334061+0000","last_fullsized":"2024-09-17T03:14:15.334061+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:39:18.278040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391218+0000","last_change":"2024-09-17T03:14:13.321965+0000","last_active":"2024-09-17T03:14:16.391218+0000","last_peered":"2024-09-17T03:14:16.391218+0000","last_clean":"2024-09-17T03:14:16.391218+0000","last_became_active":"2024-09-17T03:14:13.321616+0000","last_became_peered":"2024-09-17T03:14:13.321616+0000","last_unstale":"2024-09-17T03:14:16.391218+0000","last_undegraded":"2024-09-17T03:14:16.391218+0000","last_fullsized":"2024-09-17T03:14:16.391218+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:05:46.263336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392183+0000","last_change":"2024-09-17T03:14:14.348468+0000","last_active":"2024-09-17T03:14:16.392183+0000","last_peered":"2024-09-17T03:14:16.392183+0000","last_clean":"2024-09-17T03:14:16.392183+0000","last_became_active":"2024-09-17T03:14:14.348144+0000","last_became_peered":"2024-09-17T03:14:14.348144+0000","last_unstale":"2024-09-17T03:14:16.392183+0000","last_undegraded":"2024-09-17T03:14:16.392183+0000","last_fullsized":"2024-09-17T03:14:16.392183+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:48:14.649846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332338+0000","last_change":"2024-09-17T03:14:13.329841+0000","last_active":"2024-09-17T03:14:15.332338+0000","last_peered":"2024-09-17T03:14:15.332338+0000","last_clean":"2024-09-17T03:14:15.332338+0000","last_became_active":"2024-09-17T03:14:13.329154+0000","last_became_peered":"2024-09-17T03:14:13.329154+0000","last_unstale":"2024-09-17T03:14:15.332338+0000","last_undegraded":"2024-09-17T03:14:15.332338+0000","last_fullsized":"2024-09-17T03:14:15.332338+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:39.634907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.335175+0000","last_change":"2024-09-17T03:14:14.345392+0000","last_active":"2024-09-17T03:14:15.335175+0000","last_peered":"2024-09-17T03:14:15.335175+0000","last_clean":"2024-09-17T03:14:15.335175+0000","last_became_active":"2024-09-17T03:14:14.345073+0000","last_became_peered":"2024-09-17T03:14:14.345073+0000","last_unstale":"2024-09-17T03:14:15.335175+0000","last_undegraded":"2024-09-17T03:14:15.335175+0000","last_fullsized":"2024-09-17T03:14:15.335175+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:37:04.528125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391685+0000","last_change":"2024-09-17T03:14:13.313377+0000","last_active":"2024-09-17T03:14:16.391685+0000","last_peered":"2024-09-17T03:14:16.391685+0000","last_clean":"2024-09-17T03:14:16.391685+0000","last_became_active":"2024-09-17T03:14:13.312848+0000","last_became_peered":"2024-09-17T03:14:13.312848+0000","last_unstale":"2024-09-17T03:14:16.391685+0000","last_undegraded":"2024-09-17T03:14:16.391685+0000","last_fullsized":"2024-09-17T03:14:16.391685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:03:00.266996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333208+0000","last_change":"2024-09-17T03:14:14.344920+0000","last_active":"2024-09-17T03:14:15.333208+0000","last_peered":"2024-09-17T03:14:15.333208+0000","last_clean":"2024-09-17T03:14:15.333208+0000","last_became_active":"2024-09-17T03:14:14.344570+0000","last_became_peered":"2024-09-17T03:14:14.344570+0000","last_unstale":"2024-09-17T03:14:15.333208+0000","last_undegraded":"2024-09-17T03:14:15.333208+0000","last_fullsized":"2024-09-17T03:14:15.333208+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:03.086199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391710+0000","last_change":"2024-09-17T03:14:13.322066+0000","last_active":"2024-09-17T03:14:16.391710+0000","last_peered":"2024-09-17T03:14:16.391710+0000","last_clean":"2024-09-17T03:14:16.391710+0000","last_became_active":"2024-09-17T03:14:13.321679+0000","last_became_peered":"2024-09-17T03:14:13.321679+0000","last_unstale":"2024-09-17T03:14:16.391710+0000","last_undegraded":"2024-09-17T03:14:16.391710+0000","last_fullsized":"2024-09-17T03:14:16.391710+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:55.304571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333004+0000","last_change":"2024-09-17T03:14:14.337601+0000","last_active":"2024-09-17T03:14:15.333004+0000","last_peered":"2024-09-17T03:14:15.333004+0000","last_clean":"2024-09-17T03:14:15.333004+0000","last_became_active":"2024-09-17T03:14:14.336457+0000","last_became_peered":"2024-09-17T03:14:14.336457+0000","last_unstale":"2024-09-17T03:14:15.333004+0000","last_undegraded":"2024-09-17T03:14:15.333004+0000","last_fullsized":"2024-09-17T03:14:15.333004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:53:37.876227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391686+0000","last_change":"2024-09-17T03:14:13.318626+0000","last_active":"2024-09-17T03:14:16.391686+0000","last_peered":"2024-09-17T03:14:16.391686+0000","last_clean":"2024-09-17T03:14:16.391686+0000","last_became_active":"2024-09-17T03:14:13.318302+0000","last_became_peered":"2024-09-17T03:14:13.318302+0000","last_unstale":"2024-09-17T03:14:16.391686+0000","last_undegraded":"2024-09-17T03:14:16.391686+0000","last_fullsized":"2024-09-17T03:14:16.391686+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:53.797340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392030+0000","last_change":"2024-09-17T03:14:14.344640+0000","last_active":"2024-09-17T03:14:16.392030+0000","last_peered":"2024-09-17T03:14:16.392030+0000","last_clean":"2024-09-17T03:14:16.392030+0000","last_became_active":"2024-09-17T03:14:14.344288+0000","last_became_peered":"2024-09-17T03:14:14.344288+0000","last_unstale":"2024-09-17T03:14:16.392030+0000","last_undegraded":"2024-09-17T03:14:16.392030+0000","last_fullsized":"2024-09-17T03:14:16.392030+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:55:42.826482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333916+0000","last_change":"2024-09-17T03:14:13.317055+0000","last_active":"2024-09-17T03:14:15.333916+0000","last_peered":"2024-09-17T03:14:15.333916+0000","last_clean":"2024-09-17T03:14:15.333916+0000","last_became_active":"2024-09-17T03:14:13.316736+0000","last_became_peered":"2024-09-17T03:14:13.316736+0000","last_unstale":"2024-09-17T03:14:15.333916+0000","last_undegraded":"2024-09-17T03:14:15.333916+0000","last_fullsized":"2024-09-17T03:14:15.333916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:13.543618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333254+0000","last_change":"2024-09-17T03:14:14.341872+0000","last_active":"2024-09-17T03:14:15.333254+0000","last_peered":"2024-09-17T03:14:15.333254+0000","last_clean":"2024-09-17T03:14:15.333254+0000","last_became_active":"2024-09-17T03:14:14.341555+0000","last_became_peered":"2024-09-17T03:14:14.341555+0000","last_unstale":"2024-09-17T03:14:15.333254+0000","last_undegraded":"2024-09-17T03:14:15.333254+0000","last_fullsized":"2024-09-17T03:14:15.333254+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:14.208665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332452+0000","last_change":"2024-09-17T03:14:13.325444+0000","last_active":"2024-09-17T03:14:15.332452+0000","last_peered":"2024-09-17T03:14:15.332452+0000","last_clean":"2024-09-17T03:14:15.332452+0000","last_became_active":"2024-09-17T03:14:13.325121+0000","last_became_peered":"2024-09-17T03:14:13.325121+0000","last_unstale":"2024-09-17T03:14:15.332452+0000","last_undegraded":"2024-09-17T03:14:15.332452+0000","last_fullsized":"2024-09-17T03:14:15.332452+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:55:19.372118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.390899+0000","last_change":"2024-09-17T03:14:14.341354+0000","last_active":"2024-09-17T03:14:16.390899+0000","last_peered":"2024-09-17T03:14:16.390899+0000","last_clean":"2024-09-17T03:14:16.390899+0000","last_became_active":"2024-09-17T03:14:14.341045+0000","last_became_peered":"2024-09-17T03:14:14.341045+0000","last_unstale":"2024-09-17T03:14:16.390899+0000","last_undegraded":"2024-09-17T03:14:16.390899+0000","last_fullsized":"2024-09-17T03:14:16.390899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:01:30.375807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333084+0000","last_change":"2024-09-17T03:14:13.321317+0000","last_active":"2024-09-17T03:14:15.333084+0000","last_peered":"2024-09-17T03:14:15.333084+0000","last_clean":"2024-09-17T03:14:15.333084+0000","last_became_active":"2024-09-17T03:14:13.320995+0000","last_became_peered":"2024-09-17T03:14:13.320995+0000","last_unstale":"2024-09-17T03:14:15.333084+0000","last_undegraded":"2024-09-17T03:14:15.333084+0000","last_fullsized":"2024-09-17T03:14:15.333084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:48:56.906875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333598+0000","last_change":"2024-09-17T03:14:14.342775+0000","last_active":"2024-09-17T03:14:15.333598+0000","last_peered":"2024-09-17T03:14:15.333598+0000","last_clean":"2024-09-17T03:14:15.333598+0000","last_became_active":"2024-09-17T03:14:14.342463+0000","last_became_peered":"2024-09-17T03:14:14.342463+0000","last_unstale":"2024-09-17T03:14:15.333598+0000","last_undegraded":"2024-09-17T03:14:15.333598+0000","last_fullsized":"2024-09-17T03:14:15.333598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:29:58.176940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333151+0000","last_change":"2024-09-17T03:14:13.324555+0000","last_active":"2024-09-17T03:14:15.333151+0000","last_peered":"2024-09-17T03:14:15.333151+0000","last_clean":"2024-09-17T03:14:15.333151+0000","last_became_active":"2024-09-17T03:14:13.323910+0000","last_became_peered":"2024-09-17T03:14:13.323910+0000","last_unstale":"2024-09-17T03:14:15.333151+0000","last_undegraded":"2024-09-17T03:14:15.333151+0000","last_fullsized":"2024-09-17T03:14:15.333151+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:55:48.923806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392232+0000","last_change":"2024-09-17T03:14:14.334852+0000","last_active":"2024-09-17T03:14:16.392232+0000","last_peered":"2024-09-17T03:14:16.392232+0000","last_clean":"2024-09-17T03:14:16.392232+0000","last_became_active":"2024-09-17T03:14:14.334469+0000","last_became_peered":"2024-09-17T03:14:14.334469+0000","last_unstale":"2024-09-17T03:14:16.392232+0000","last_undegraded":"2024-09-17T03:14:16.392232+0000","last_fullsized":"2024-09-17T03:14:16.392232+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:25:14.548392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333987+0000","last_change":"2024-09-17T03:14:13.322488+0000","last_active":"2024-09-17T03:14:15.333987+0000","last_peered":"2024-09-17T03:14:15.333987+0000","last_clean":"2024-09-17T03:14:15.333987+0000","last_became_active":"2024-09-17T03:14:13.322170+0000","last_became_peered":"2024-09-17T03:14:13.322170+0000","last_unstale":"2024-09-17T03:14:15.333987+0000","last_undegraded":"2024-09-17T03:14:15.333987+0000","last_fullsized":"2024-09-17T03:14:15.333987+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:13.679632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333094+0000","last_change":"2024-09-17T03:14:14.336281+0000","last_active":"2024-09-17T03:14:15.333094+0000","last_peered":"2024-09-17T03:14:15.333094+0000","last_clean":"2024-09-17T03:14:15.333094+0000","last_became_active":"2024-09-17T03:14:14.335986+0000","last_became_peered":"2024-09-17T03:14:14.335986+0000","last_unstale":"2024-09-17T03:14:15.333094+0000","last_undegraded":"2024-09-17T03:14:15.333094+0000","last_fullsized":"2024-09-17T03:14:15.333094+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:10:18.826069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334554+0000","last_change":"2024-09-17T03:14:13.318667+0000","last_active":"2024-09-17T03:14:15.334554+0000","last_peered":"2024-09-17T03:14:15.334554+0000","last_clean":"2024-09-17T03:14:15.334554+0000","last_became_active":"2024-09-17T03:14:13.318364+0000","last_became_peered":"2024-09-17T03:14:13.318364+0000","last_unstale":"2024-09-17T03:14:15.334554+0000","last_undegraded":"2024-09-17T03:14:15.334554+0000","last_fullsized":"2024-09-17T03:14:15.334554+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:48:55.719231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333416+0000","last_change":"2024-09-17T03:14:14.334875+0000","last_active":"2024-09-17T03:14:15.333416+0000","last_peered":"2024-09-17T03:14:15.333416+0000","last_clean":"2024-09-17T03:14:15.333416+0000","last_became_active":"2024-09-17T03:14:14.334511+0000","last_became_peered":"2024-09-17T03:14:14.334511+0000","last_unstale":"2024-09-17T03:14:15.333416+0000","last_undegraded":"2024-09-17T03:14:15.333416+0000","last_fullsized":"2024-09-17T03:14:15.333416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:46:26.597012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392173+0000","last_change":"2024-09-17T03:14:13.330509+0000","last_active":"2024-09-17T03:14:16.392173+0000","last_peered":"2024-09-17T03:14:16.392173+0000","last_clean":"2024-09-17T03:14:16.392173+0000","last_became_active":"2024-09-17T03:14:13.330197+0000","last_became_peered":"2024-09-17T03:14:13.330197+0000","last_unstale":"2024-09-17T03:14:16.392173+0000","last_undegraded":"2024-09-17T03:14:16.392173+0000","last_fullsized":"2024-09-17T03:14:16.392173+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:53:53.797340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334097+0000","last_change":"2024-09-17T03:14:14.345943+0000","last_active":"2024-09-17T03:14:15.334097+0000","last_peered":"2024-09-17T03:14:15.334097+0000","last_clean":"2024-09-17T03:14:15.334097+0000","last_became_active":"2024-09-17T03:14:14.345637+0000","last_became_peered":"2024-09-17T03:14:14.345637+0000","last_unstale":"2024-09-17T03:14:15.334097+0000","last_undegraded":"2024-09-17T03:14:15.334097+0000","last_fullsized":"2024-09-17T03:14:15.334097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:23.070846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.390671+0000","last_change":"2024-09-17T03:14:04.862167+0000","last_active":"2024-09-17T03:14:16.390671+0000","last_peered":"2024-09-17T03:14:16.390671+0000","last_clean":"2024-09-17T03:14:16.390671+0000","last_became_active":"2024-09-17T03:14:04.861731+0000","last_became_peered":"2024-09-17T03:14:04.861731+0000","last_unstale":"2024-09-17T03:14:16.390671+0000","last_undegraded":"2024-09-17T03:14:16.390671+0000","last_fullsized":"2024-09-17T03:14:16.390671+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_clean_scrub_stamp":"2024-09-17T03:14:03.847721+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:32.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333439+0000","last_change":"2024-09-17T03:14:13.318977+0000","last_active":"2024-09-17T03:14:15.333439+0000","last_peered":"2024-09-17T03:14:15.333439+0000","last_clean":"2024-09-17T03:14:15.333439+0000","last_became_active":"2024-09-17T03:14:13.318294+0000","last_became_peered":"2024-09-17T03:14:13.318294+0000","last_unstale":"2024-09-17T03:14:15.333439+0000","last_undegraded":"2024-09-17T03:14:15.333439+0000","last_fullsized":"2024-09-17T03:14:15.333439+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:57:39.634907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333413+0000","last_change":"2024-09-17T03:14:14.331745+0000","last_active":"2024-09-17T03:14:15.333413+0000","last_peered":"2024-09-17T03:14:15.333413+0000","last_clean":"2024-09-17T03:14:15.333413+0000","last_became_active":"2024-09-17T03:14:14.331207+0000","last_became_peered":"2024-09-17T03:14:14.331207+0000","last_unstale":"2024-09-17T03:14:15.333413+0000","last_undegraded":"2024-09-17T03:14:15.333413+0000","last_fullsized":"2024-09-17T03:14:15.333413+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:53:48.931242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332808+0000","last_change":"2024-09-17T03:14:13.324555+0000","last_active":"2024-09-17T03:14:15.332808+0000","last_peered":"2024-09-17T03:14:15.332808+0000","last_clean":"2024-09-17T03:14:15.332808+0000","last_became_active":"2024-09-17T03:14:13.324214+0000","last_became_peered":"2024-09-17T03:14:13.324214+0000","last_unstale":"2024-09-17T03:14:15.332808+0000","last_undegraded":"2024-09-17T03:14:15.332808+0000","last_fullsized":"2024-09-17T03:14:15.332808+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:29:13.693714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391471+0000","last_change":"2024-09-17T03:14:14.339552+0000","last_active":"2024-09-17T03:14:16.391471+0000","last_peered":"2024-09-17T03:14:16.391471+0000","last_clean":"2024-09-17T03:14:16.391471+0000","last_became_active":"2024-09-17T03:14:14.339232+0000","last_became_peered":"2024-09-17T03:14:14.339232+0000","last_unstale":"2024-09-17T03:14:16.391471+0000","last_undegraded":"2024-09-17T03:14:16.391471+0000","last_fullsized":"2024-09-17T03:14:16.391471+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:31.035937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391696+0000","last_change":"2024-09-17T03:14:13.320582+0000","last_active":"2024-09-17T03:14:16.391696+0000","last_peered":"2024-09-17T03:14:16.391696+0000","last_clean":"2024-09-17T03:14:16.391696+0000","last_became_active":"2024-09-17T03:14:13.320217+0000","last_became_peered":"2024-09-17T03:14:13.320217+0000","last_unstale":"2024-09-17T03:14:16.391696+0000","last_undegraded":"2024-09-17T03:14:16.391696+0000","last_fullsized":"2024-09-17T03:14:16.391696+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:34.405796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334392+0000","last_change":"2024-09-17T03:14:14.334667+0000","last_active":"2024-09-17T03:14:15.334392+0000","last_peered":"2024-09-17T03:14:15.334392+0000","last_clean":"2024-09-17T03:14:15.334392+0000","last_became_active":"2024-09-17T03:14:14.333112+0000","last_became_peered":"2024-09-17T03:14:14.333112+0000","last_unstale":"2024-09-17T03:14:15.334392+0000","last_undegraded":"2024-09-17T03:14:15.334392+0000","last_fullsized":"2024-09-17T03:14:15.334392+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:04:17.642423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334025+0000","last_change":"2024-09-17T03:14:13.327887+0000","last_active":"2024-09-17T03:14:15.334025+0000","last_peered":"2024-09-17T03:14:15.334025+0000","last_clean":"2024-09-17T03:14:15.334025+0000","last_became_active":"2024-09-17T03:14:13.327514+0000","last_became_peered":"2024-09-17T03:14:13.327514+0000","last_unstale":"2024-09-17T03:14:15.334025+0000","last_undegraded":"2024-09-17T03:14:15.334025+0000","last_fullsized":"2024-09-17T03:14:15.334025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:10:41.615501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333005+0000","last_change":"2024-09-17T03:14:14.338404+0000","last_active":"2024-09-17T03:14:15.333005+0000","last_peered":"2024-09-17T03:14:15.333005+0000","last_clean":"2024-09-17T03:14:15.333005+0000","last_became_active":"2024-09-17T03:14:14.338120+0000","last_became_peered":"2024-09-17T03:14:14.338120+0000","last_unstale":"2024-09-17T03:14:15.333005+0000","last_undegraded":"2024-09-17T03:14:15.333005+0000","last_fullsized":"2024-09-17T03:14:15.333005+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:59:17.576471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334665+0000","last_change":"2024-09-17T03:14:13.324534+0000","last_active":"2024-09-17T03:14:15.334665+0000","last_peered":"2024-09-17T03:14:15.334665+0000","last_clean":"2024-09-17T03:14:15.334665+0000","last_became_active":"2024-09-17T03:14:13.324173+0000","last_became_peered":"2024-09-17T03:14:13.324173+0000","last_unstale":"2024-09-17T03:14:15.334665+0000","last_undegraded":"2024-09-17T03:14:15.334665+0000","last_fullsized":"2024-09-17T03:14:15.334665+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:27:20.601544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391770+0000","last_change":"2024-09-17T03:14:14.341535+0000","last_active":"2024-09-17T03:14:16.391770+0000","last_peered":"2024-09-17T03:14:16.391770+0000","last_clean":"2024-09-17T03:14:16.391770+0000","last_became_active":"2024-09-17T03:14:14.339526+0000","last_became_peered":"2024-09-17T03:14:14.339526+0000","last_unstale":"2024-09-17T03:14:16.391770+0000","last_undegraded":"2024-09-17T03:14:16.391770+0000","last_fullsized":"2024-09-17T03:14:16.391770+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:45.268231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334336+0000","last_change":"2024-09-17T03:14:13.321581+0000","last_active":"2024-09-17T03:14:15.334336+0000","last_peered":"2024-09-17T03:14:15.334336+0000","last_clean":"2024-09-17T03:14:15.334336+0000","last_became_active":"2024-09-17T03:14:13.321238+0000","last_became_peered":"2024-09-17T03:14:13.321238+0000","last_unstale":"2024-09-17T03:14:15.334336+0000","last_undegraded":"2024-09-17T03:14:15.334336+0000","last_fullsized":"2024-09-17T03:14:15.334336+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:57.200786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392107+0000","last_change":"2024-09-17T03:14:14.343574+0000","last_active":"2024-09-17T03:14:16.392107+0000","last_peered":"2024-09-17T03:14:16.392107+0000","last_clean":"2024-09-17T03:14:16.392107+0000","last_became_active":"2024-09-17T03:14:14.342586+0000","last_became_peered":"2024-09-17T03:14:14.342586+0000","last_unstale":"2024-09-17T03:14:16.392107+0000","last_undegraded":"2024-09-17T03:14:16.392107+0000","last_fullsized":"2024-09-17T03:14:16.392107+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:53:20.317357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334149+0000","last_change":"2024-09-17T03:14:13.321756+0000","last_active":"2024-09-17T03:14:15.334149+0000","last_peered":"2024-09-17T03:14:15.334149+0000","last_clean":"2024-09-17T03:14:15.334149+0000","last_became_active":"2024-09-17T03:14:13.321392+0000","last_became_peered":"2024-09-17T03:14:13.321392+0000","last_unstale":"2024-09-17T03:14:15.334149+0000","last_undegraded":"2024-09-17T03:14:15.334149+0000","last_fullsized":"2024-09-17T03:14:15.334149+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:00:33.594511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333517+0000","last_change":"2024-09-17T03:14:14.341909+0000","last_active":"2024-09-17T03:14:15.333517+0000","last_peered":"2024-09-17T03:14:15.333517+0000","last_clean":"2024-09-17T03:14:15.333517+0000","last_became_active":"2024-09-17T03:14:14.341571+0000","last_became_peered":"2024-09-17T03:14:14.341571+0000","last_unstale":"2024-09-17T03:14:15.333517+0000","last_undegraded":"2024-09-17T03:14:15.333517+0000","last_fullsized":"2024-09-17T03:14:15.333517+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:41.776202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334199+0000","last_change":"2024-09-17T03:14:13.329358+0000","last_active":"2024-09-17T03:14:15.334199+0000","last_peered":"2024-09-17T03:14:15.334199+0000","last_clean":"2024-09-17T03:14:15.334199+0000","last_became_active":"2024-09-17T03:14:13.328836+0000","last_became_peered":"2024-09-17T03:14:13.328836+0000","last_unstale":"2024-09-17T03:14:15.334199+0000","last_undegraded":"2024-09-17T03:14:15.334199+0000","last_fullsized":"2024-09-17T03:14:15.334199+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:44.263456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333819+0000","last_change":"2024-09-17T03:14:14.340502+0000","last_active":"2024-09-17T03:14:15.333819+0000","last_peered":"2024-09-17T03:14:15.333819+0000","last_clean":"2024-09-17T03:14:15.333819+0000","last_became_active":"2024-09-17T03:14:14.340215+0000","last_became_peered":"2024-09-17T03:14:14.340215+0000","last_unstale":"2024-09-17T03:14:15.333819+0000","last_undegraded":"2024-09-17T03:14:15.333819+0000","last_fullsized":"2024-09-17T03:14:15.333819+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:45.283163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.332057+0000","last_change":"2024-09-17T03:14:13.324045+0000","last_active":"2024-09-17T03:14:15.332057+0000","last_peered":"2024-09-17T03:14:15.332057+0000","last_clean":"2024-09-17T03:14:15.332057+0000","last_became_active":"2024-09-17T03:14:13.323025+0000","last_became_peered":"2024-09-17T03:14:13.323025+0000","last_unstale":"2024-09-17T03:14:15.332057+0000","last_undegraded":"2024-09-17T03:14:15.332057+0000","last_fullsized":"2024-09-17T03:14:15.332057+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:10:02.157890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333680+0000","last_change":"2024-09-17T03:14:14.346858+0000","last_active":"2024-09-17T03:14:15.333680+0000","last_peered":"2024-09-17T03:14:15.333680+0000","last_clean":"2024-09-17T03:14:15.333680+0000","last_became_active":"2024-09-17T03:14:14.346576+0000","last_became_peered":"2024-09-17T03:14:14.346576+0000","last_unstale":"2024-09-17T03:14:15.333680+0000","last_undegraded":"2024-09-17T03:14:15.333680+0000","last_fullsized":"2024-09-17T03:14:15.333680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:36:38.080821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334132+0000","last_change":"2024-09-17T03:14:13.319558+0000","last_active":"2024-09-17T03:14:15.334132+0000","last_peered":"2024-09-17T03:14:15.334132+0000","last_clean":"2024-09-17T03:14:15.334132+0000","last_became_active":"2024-09-17T03:14:13.319236+0000","last_became_peered":"2024-09-17T03:14:13.319236+0000","last_unstale":"2024-09-17T03:14:15.334132+0000","last_undegraded":"2024-09-17T03:14:15.334132+0000","last_fullsized":"2024-09-17T03:14:15.334132+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:25:39.260408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333163+0000","last_change":"2024-09-17T03:14:14.329694+0000","last_active":"2024-09-17T03:14:15.333163+0000","last_peered":"2024-09-17T03:14:15.333163+0000","last_clean":"2024-09-17T03:14:15.333163+0000","last_became_active":"2024-09-17T03:14:14.329289+0000","last_became_peered":"2024-09-17T03:14:14.329289+0000","last_unstale":"2024-09-17T03:14:15.333163+0000","last_undegraded":"2024-09-17T03:14:15.333163+0000","last_fullsized":"2024-09-17T03:14:15.333163+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:59.114330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392121+0000","last_change":"2024-09-17T03:14:13.323443+0000","last_active":"2024-09-17T03:14:16.392121+0000","last_peered":"2024-09-17T03:14:16.392121+0000","last_clean":"2024-09-17T03:14:16.392121+0000","last_became_active":"2024-09-17T03:14:13.323047+0000","last_became_peered":"2024-09-17T03:14:13.323047+0000","last_unstale":"2024-09-17T03:14:16.392121+0000","last_undegraded":"2024-09-17T03:14:16.392121+0000","last_fullsized":"2024-09-17T03:14:16.392121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:59:34.405796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.392246+0000","last_change":"2024-09-17T03:14:14.336168+0000","last_active":"2024-09-17T03:14:16.392246+0000","last_peered":"2024-09-17T03:14:16.392246+0000","last_clean":"2024-09-17T03:14:16.392246+0000","last_became_active":"2024-09-17T03:14:14.335860+0000","last_became_peered":"2024-09-17T03:14:14.335860+0000","last_unstale":"2024-09-17T03:14:16.392246+0000","last_undegraded":"2024-09-17T03:14:16.392246+0000","last_fullsized":"2024-09-17T03:14:16.392246+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:02:40.435687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333326+0000","last_change":"2024-09-17T03:14:13.319349+0000","last_active":"2024-09-17T03:14:15.333326+0000","last_peered":"2024-09-17T03:14:15.333326+0000","last_clean":"2024-09-17T03:14:15.333326+0000","last_became_active":"2024-09-17T03:14:13.318677+0000","last_became_peered":"2024-09-17T03:14:13.318677+0000","last_unstale":"2024-09-17T03:14:15.333326+0000","last_undegraded":"2024-09-17T03:14:15.333326+0000","last_fullsized":"2024-09-17T03:14:15.333326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:55:19.372118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333793+0000","last_change":"2024-09-17T03:14:14.343651+0000","last_active":"2024-09-17T03:14:15.333793+0000","last_peered":"2024-09-17T03:14:15.333793+0000","last_clean":"2024-09-17T03:14:15.333793+0000","last_became_active":"2024-09-17T03:14:14.343219+0000","last_became_peered":"2024-09-17T03:14:14.343219+0000","last_unstale":"2024-09-17T03:14:15.333793+0000","last_undegraded":"2024-09-17T03:14:15.333793+0000","last_fullsized":"2024-09-17T03:14:15.333793+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:13:32.656876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333968+0000","last_change":"2024-09-17T03:14:14.347881+0000","last_active":"2024-09-17T03:14:15.333968+0000","last_peered":"2024-09-17T03:14:15.333968+0000","last_clean":"2024-09-17T03:14:15.333968+0000","last_became_active":"2024-09-17T03:14:14.346360+0000","last_became_peered":"2024-09-17T03:14:14.346360+0000","last_unstale":"2024-09-17T03:14:15.333968+0000","last_undegraded":"2024-09-17T03:14:15.333968+0000","last_fullsized":"2024-09-17T03:14:15.333968+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:41:21.993789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333484+0000","last_change":"2024-09-17T03:14:13.329489+0000","last_active":"2024-09-17T03:14:15.333484+0000","last_peered":"2024-09-17T03:14:15.333484+0000","last_clean":"2024-09-17T03:14:15.333484+0000","last_became_active":"2024-09-17T03:14:13.329141+0000","last_became_peered":"2024-09-17T03:14:13.329141+0000","last_unstale":"2024-09-17T03:14:15.333484+0000","last_undegraded":"2024-09-17T03:14:15.333484+0000","last_fullsized":"2024-09-17T03:14:15.333484+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:38:44.263456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:16.391814+0000","last_change":"2024-09-17T03:14:14.337492+0000","last_active":"2024-09-17T03:14:16.391814+0000","last_peered":"2024-09-17T03:14:16.391814+0000","last_clean":"2024-09-17T03:14:16.391814+0000","last_became_active":"2024-09-17T03:14:14.337160+0000","last_became_peered":"2024-09-17T03:14:14.337160+0000","last_unstale":"2024-09-17T03:14:16.391814+0000","last_undegraded":"2024-09-17T03:14:16.391814+0000","last_fullsized":"2024-09-17T03:14:16.391814+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:31.035937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333395+0000","last_change":"2024-09-17T03:14:13.323396+0000","last_active":"2024-09-17T03:14:15.333395+0000","last_peered":"2024-09-17T03:14:15.333395+0000","last_clean":"2024-09-17T03:14:15.333395+0000","last_became_active":"2024-09-17T03:14:13.322911+0000","last_became_peered":"2024-09-17T03:14:13.322911+0000","last_unstale":"2024-09-17T03:14:15.333395+0000","last_undegraded":"2024-09-17T03:14:15.333395+0000","last_fullsized":"2024-09-17T03:14:15.333395+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:36.650878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.335094+0000","last_change":"2024-09-17T03:14:14.340359+0000","last_active":"2024-09-17T03:14:15.335094+0000","last_peered":"2024-09-17T03:14:15.335094+0000","last_clean":"2024-09-17T03:14:15.335094+0000","last_became_active":"2024-09-17T03:14:14.340060+0000","last_became_peered":"2024-09-17T03:14:14.340060+0000","last_unstale":"2024-09-17T03:14:15.335094+0000","last_undegraded":"2024-09-17T03:14:15.335094+0000","last_fullsized":"2024-09-17T03:14:15.335094+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:59:36.040787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.334845+0000","last_change":"2024-09-17T03:14:13.317875+0000","last_active":"2024-09-17T03:14:15.334845+0000","last_peered":"2024-09-17T03:14:15.334845+0000","last_clean":"2024-09-17T03:14:15.334845+0000","last_became_active":"2024-09-17T03:14:13.317521+0000","last_became_peered":"2024-09-17T03:14:13.317521+0000","last_unstale":"2024-09-17T03:14:15.334845+0000","last_undegraded":"2024-09-17T03:14:15.334845+0000","last_fullsized":"2024-09-17T03:14:15.334845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:39:05.715850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.335048+0000","last_change":"2024-09-17T03:14:14.338400+0000","last_active":"2024-09-17T03:14:15.335048+0000","last_peered":"2024-09-17T03:14:15.335048+0000","last_clean":"2024-09-17T03:14:15.335048+0000","last_became_active":"2024-09-17T03:14:14.338003+0000","last_became_peered":"2024-09-17T03:14:14.338003+0000","last_unstale":"2024-09-17T03:14:15.335048+0000","last_undegraded":"2024-09-17T03:14:15.335048+0000","last_fullsized":"2024-09-17T03:14:15.335048+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:13.298793+0000","last_clean_scrub_stamp":"2024-09-17T03:14:13.298793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:57:32.024812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-17T03:14:15.333702+0000","last_change":"2024-09-17T03:14:13.324341+0000","last_active":"2024-09-17T03:14:15.333702+0000","last_peered":"2024-09-17T03:14:15.333702+0000","last_clean":"2024-09-17T03:14:15.333702+0000","last_became_active":"2024-09-17T03:14:13.323959+0000","last_became_peered":"2024-09-17T03:14:13.323959+0000","last_unstale":"2024-09-17T03:14:15.333702+0000","last_undegraded":"2024-09-17T03:14:15.333702+0000","last_fullsized":"2024-09-17T03:14:15.333702+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:12.294019+0000","last_clean_scrub_stamp":"2024-09-17T03:14:12.294019+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:18:06.456762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":241664,"data_stored":82231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":82107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27844,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343996,"statfs":{"total":96636764160,"available":96608251904,"internally_reserved":0,"allocated":835584,"data_stored":672567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509449,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":82689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":245760,"data_stored":83719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":672545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":82163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":253952,"data_stored":84391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:14:27.002 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T03:14:27.002 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-17T03:14:27.002 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-17T03:14:27.002 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-17T03:14:27.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.150+0000 7f032d7cc640 1 -- 172.21.15.87:0/2817898523 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0328160d30 msgr2=0x7f0328161140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:27.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.150+0000 7f032d7cc640 1 --2- 172.21.15.87:0/2817898523 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0328160d30 0x7f0328161140 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f032808c870 tx=0x7f032002a980 comp rx=0 tx=0).stop 2024-09-17T03:14:27.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.150+0000 7f032d7cc640 1 -- 172.21.15.87:0/2817898523 shutdown_connections 2024-09-17T03:14:27.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.150+0000 7f032d7cc640 1 --2- 172.21.15.87:0/2817898523 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0328160d30 0x7f0328161140 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:27.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.150+0000 7f032d7cc640 1 --2- 172.21.15.87:0/2817898523 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f03280d1400 0x7f03281607f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:27.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.150+0000 7f032d7cc640 1 --2- 172.21.15.87:0/2817898523 >> v2:172.21.15.149:3300/0 conn(0x7f03280b33d0 0x7f03280d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:27.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.150+0000 7f032d7cc640 1 -- 172.21.15.87:0/2817898523 >> 172.21.15.87:0/2817898523 conn(0x7f032809d700 msgr2=0x7f03280d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:27.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.150+0000 7f032d7cc640 1 -- 172.21.15.87:0/2817898523 shutdown_connections 2024-09-17T03:14:27.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.150+0000 7f032d7cc640 1 -- 172.21.15.87:0/2817898523 wait complete. 2024-09-17T03:14:27.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.152+0000 7f032d7cc640 1 Processor -- start 2024-09-17T03:14:27.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.152+0000 7f032d7cc640 1 -- start start 2024-09-17T03:14:27.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.153+0000 7f032d7cc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03280b33d0 0x7f032809e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:27.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.154+0000 7f032d7cc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03280d1400 0x7f0328156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:27.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.154+0000 7f032d7cc640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0328160d30 0x7f032809fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:27.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.154+0000 7f032d7cc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0328161800 con 0x7f03280d1400 2024-09-17T03:14:27.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.154+0000 7f032d7cc640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0328161680 con 0x7f03280b33d0 2024-09-17T03:14:27.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.154+0000 7f032d7cc640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0328161980 con 0x7f0328160d30 2024-09-17T03:14:27.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.154+0000 7f031e7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03280d1400 0x7f0328156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:27.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.154+0000 7f031e7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03280d1400 0x7f0328156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50894/0 (socket says 172.21.15.87:50894) 2024-09-17T03:14:27.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.154+0000 7f031e7fc640 1 -- 172.21.15.87:0/680879848 learned_addr learned my addr 172.21.15.87:0/680879848 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:27.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.154+0000 7f03277fe640 1 --2- 172.21.15.87:0/680879848 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0328160d30 0x7f032809fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:27.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.154+0000 7f0326ffd640 1 --2- 172.21.15.87:0/680879848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03280b33d0 0x7f032809e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:27.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.154+0000 7f031e7fc640 1 -- 172.21.15.87:0/680879848 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0328160d30 msgr2=0x7f032809fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:27.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.154+0000 7f031e7fc640 1 --2- 172.21.15.87:0/680879848 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0328160d30 0x7f032809fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:27.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.154+0000 7f031e7fc640 1 -- 172.21.15.87:0/680879848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03280b33d0 msgr2=0x7f032809e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:27.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.154+0000 7f031e7fc640 1 --2- 172.21.15.87:0/680879848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03280b33d0 0x7f032809e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:27.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.154+0000 7f031e7fc640 1 -- 172.21.15.87:0/680879848 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032809ea50 con 0x7f03280d1400 2024-09-17T03:14:27.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.154+0000 7f0326ffd640 1 --2- 172.21.15.87:0/680879848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03280b33d0 0x7f032809e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:27.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.154+0000 7f031e7fc640 1 --2- 172.21.15.87:0/680879848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03280d1400 0x7f0328156910 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f031000d6d0 tx=0x7f031000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:27.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.154+0000 7f031ffff640 1 -- 172.21.15.87:0/680879848 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310017070 con 0x7f03280d1400 2024-09-17T03:14:27.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.154+0000 7f031ffff640 1 -- 172.21.15.87:0/680879848 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0310005330 con 0x7f03280d1400 2024-09-17T03:14:27.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.154+0000 7f032d7cc640 1 -- 172.21.15.87:0/680879848 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f032809ed30 con 0x7f03280d1400 2024-09-17T03:14:27.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.154+0000 7f031ffff640 1 -- 172.21.15.87:0/680879848 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310009820 con 0x7f03280d1400 2024-09-17T03:14:27.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.154+0000 7f032d7cc640 1 -- 172.21.15.87:0/680879848 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f032809f2f0 con 0x7f03280d1400 2024-09-17T03:14:27.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.155+0000 7f032d7cc640 1 -- 172.21.15.87:0/680879848 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f032809e7c0 con 0x7f03280d1400 2024-09-17T03:14:27.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.156+0000 7f031ffff640 1 -- 172.21.15.87:0/680879848 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f03100099c0 con 0x7f03280d1400 2024-09-17T03:14:27.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.156+0000 7f031ffff640 1 --2- 172.21.15.87:0/680879848 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0308078bc0 0x7f030807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:27.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.157+0000 7f031ffff640 1 -- 172.21.15.87:0/680879848 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f031009ac80 con 0x7f03280d1400 2024-09-17T03:14:27.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.157+0000 7f0326ffd640 1 --2- 172.21.15.87:0/680879848 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0308078bc0 0x7f030807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:27.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.157+0000 7f0326ffd640 1 --2- 172.21.15.87:0/680879848 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0308078bc0 0x7f030807b0d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0318005fd0 tx=0x7f031800b530 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:27.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.161+0000 7f031ffff640 1 -- 172.21.15.87:0/680879848 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0310062b90 con 0x7f03280d1400 2024-09-17T03:14:27.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.373+0000 7f032d7cc640 1 -- 172.21.15.87:0/680879848 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f032806de80 con 0x7f03280d1400 2024-09-17T03:14:27.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.373+0000 7f031ffff640 1 -- 172.21.15.87:0/680879848 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f03100675c0 con 0x7f03280d1400 2024-09-17T03:14:27.372 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:27.372 INFO:teuthology.orchestra.run.smithi087.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-17T03:14:27.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.375+0000 7f032d7cc640 1 -- 172.21.15.87:0/680879848 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0308078bc0 msgr2=0x7f030807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:27.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.375+0000 7f032d7cc640 1 --2- 172.21.15.87:0/680879848 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0308078bc0 0x7f030807b0d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0318005fd0 tx=0x7f031800b530 comp rx=0 tx=0).stop 2024-09-17T03:14:27.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.375+0000 7f032d7cc640 1 -- 172.21.15.87:0/680879848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03280d1400 msgr2=0x7f0328156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:27.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.375+0000 7f032d7cc640 1 --2- 172.21.15.87:0/680879848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03280d1400 0x7f0328156910 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f031000d6d0 tx=0x7f031000db90 comp rx=0 tx=0).stop 2024-09-17T03:14:27.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.375+0000 7f032d7cc640 1 -- 172.21.15.87:0/680879848 shutdown_connections 2024-09-17T03:14:27.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.375+0000 7f032d7cc640 1 --2- 172.21.15.87:0/680879848 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0308078bc0 0x7f030807b0d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:27.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.375+0000 7f032d7cc640 1 --2- 172.21.15.87:0/680879848 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0328160d30 0x7f032809fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:27.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.375+0000 7f032d7cc640 1 --2- 172.21.15.87:0/680879848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03280d1400 0x7f0328156910 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:27.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.375+0000 7f032d7cc640 1 --2- 172.21.15.87:0/680879848 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03280b33d0 0x7f032809e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:27.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.375+0000 7f032d7cc640 1 -- 172.21.15.87:0/680879848 >> 172.21.15.87:0/680879848 conn(0x7f032809d700 msgr2=0x7f03280d2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:27.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.375+0000 7f032d7cc640 1 -- 172.21.15.87:0/680879848 shutdown_connections 2024-09-17T03:14:27.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.375+0000 7f032d7cc640 1 -- 172.21.15.87:0/680879848 wait complete. 2024-09-17T03:14:27.385 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-17T03:14:27.385 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:14:27.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.499+0000 7f20c7e9b640 1 -- 172.21.15.87:0/1289139762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20c00b6600 msgr2=0x7f20c015f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:27.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.499+0000 7f20c7e9b640 1 --2- 172.21.15.87:0/1289139762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20c00b6600 0x7f20c015f100 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f20b8005fd0 tx=0x7f20b802aa90 comp rx=0 tx=0).stop 2024-09-17T03:14:27.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.499+0000 7f20c7e9b640 1 -- 172.21.15.87:0/1289139762 shutdown_connections 2024-09-17T03:14:27.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.499+0000 7f20c7e9b640 1 --2- 172.21.15.87:0/1289139762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20c00b6600 0x7f20c015f100 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:27.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.499+0000 7f20c7e9b640 1 --2- 172.21.15.87:0/1289139762 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f20c015c2d0 0x7f20c00cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:27.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.499+0000 7f20c7e9b640 1 -- 172.21.15.87:0/1289139762 >> 172.21.15.87:0/1289139762 conn(0x7f20c00d4470 msgr2=0x7f20c00d4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:27.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.499+0000 7f20c7e9b640 1 -- 172.21.15.87:0/1289139762 shutdown_connections 2024-09-17T03:14:27.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.499+0000 7f20c7e9b640 1 -- 172.21.15.87:0/1289139762 wait complete. 2024-09-17T03:14:27.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.501+0000 7f20c7e9b640 1 Processor -- start 2024-09-17T03:14:27.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.501+0000 7f20c7e9b640 1 -- start start 2024-09-17T03:14:27.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.502+0000 7f20c7e9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20c015c2d0 0x7f20c00aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:27.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.502+0000 7f20c7e9b640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f20c015ecf0 0x7f20c015ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:27.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.502+0000 7f20c7e9b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20c0150ac0 0x7f20c01f88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:27.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.502+0000 7f20c7e9b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f20c00d2db0 con 0x7f20c015c2d0 2024-09-17T03:14:27.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.502+0000 7f20c7e9b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f20c00d2c30 con 0x7f20c0150ac0 2024-09-17T03:14:27.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.502+0000 7f20c7e9b640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f20c00d2f30 con 0x7f20c015ecf0 2024-09-17T03:14:27.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.502+0000 7f20c5c37640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20c015c2d0 0x7f20c00aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:27.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.503+0000 7f20c5c37640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20c015c2d0 0x7f20c00aadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50912/0 (socket says 172.21.15.87:50912) 2024-09-17T03:14:27.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.503+0000 7f20c5c37640 1 -- 172.21.15.87:0/3622389430 learned_addr learned my addr 172.21.15.87:0/3622389430 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:27.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.503+0000 7f20c5c37640 1 -- 172.21.15.87:0/3622389430 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f20c015ecf0 msgr2=0x7f20c015ab20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T03:14:27.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.503+0000 7f20c5c37640 1 --2- 172.21.15.87:0/3622389430 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f20c015ecf0 0x7f20c015ab20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:27.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.503+0000 7f20c5c37640 1 -- 172.21.15.87:0/3622389430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20c0150ac0 msgr2=0x7f20c01f88b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T03:14:27.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.503+0000 7f20c5c37640 1 --2- 172.21.15.87:0/3622389430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20c0150ac0 0x7f20c01f88b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:27.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.503+0000 7f20c5c37640 1 -- 172.21.15.87:0/3622389430 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c00ab5b0 con 0x7f20c015c2d0 2024-09-17T03:14:27.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.503+0000 7f20c5c37640 1 --2- 172.21.15.87:0/3622389430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20c015c2d0 0x7f20c00aadf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00d7a0 tx=0x7f20bc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:27.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.503+0000 7f20b67fc640 1 -- 172.21.15.87:0/3622389430 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc017070 con 0x7f20c015c2d0 2024-09-17T03:14:27.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.503+0000 7f20b67fc640 1 -- 172.21.15.87:0/3622389430 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20bc009d20 con 0x7f20c015c2d0 2024-09-17T03:14:27.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.503+0000 7f20b67fc640 1 -- 172.21.15.87:0/3622389430 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc004280 con 0x7f20c015c2d0 2024-09-17T03:14:27.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.503+0000 7f20c7e9b640 1 -- 172.21.15.87:0/3622389430 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c01f6ea0 con 0x7f20c015c2d0 2024-09-17T03:14:27.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.503+0000 7f20c7e9b640 1 -- 172.21.15.87:0/3622389430 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c00ca820 con 0x7f20c015c2d0 2024-09-17T03:14:27.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.504+0000 7f20c7e9b640 1 -- 172.21.15.87:0/3622389430 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20c00b6b20 con 0x7f20c015c2d0 2024-09-17T03:14:27.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.505+0000 7f20b67fc640 1 -- 172.21.15.87:0/3622389430 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f20bc005020 con 0x7f20c015c2d0 2024-09-17T03:14:27.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.505+0000 7f20b67fc640 1 --2- 172.21.15.87:0/3622389430 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f20a4078bc0 0x7f20a407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:27.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.506+0000 7f20b67fc640 1 -- 172.21.15.87:0/3622389430 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f20bc09b270 con 0x7f20c015c2d0 2024-09-17T03:14:27.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.507+0000 7f20c5436640 1 --2- 172.21.15.87:0/3622389430 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f20a4078bc0 0x7f20a407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:27.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.509+0000 7f20c5436640 1 --2- 172.21.15.87:0/3622389430 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f20a4078bc0 0x7f20a407b0d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f20c01ef8a0 tx=0x7f20b00079b0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:27.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.510+0000 7f20b67fc640 1 -- 172.21.15.87:0/3622389430 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f20c00b6b20 con 0x7f20c015c2d0 2024-09-17T03:14:27.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.719+0000 7f20c7e9b640 1 -- 172.21.15.87:0/3622389430 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20c00d34c0 con 0x7f20c015c2d0 2024-09-17T03:14:27.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.719+0000 7f20b67fc640 1 -- 172.21.15.87:0/3622389430 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5217 (secure 0 0 0) 0x7f20bc020020 con 0x7f20c015c2d0 2024-09-17T03:14:27.718 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:27.718 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":6,"btime":"2024-09-17T03:14:16:330267+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4289,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/110106418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":110106418},{"type":"v1","addr":"172.21.15.149:6835","nonce":110106418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1336626873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1336626873},{"type":"v1","addr":"172.21.15.149:6837","nonce":1336626873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/666860906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":666860906},{"type":"v1","addr":"172.21.15.87:6833","nonce":666860906}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:14:13.672621+0000","modified":"2024-09-17T03:14:15.675568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4398":{"gid":4398,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/1831073338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1831073338},{"type":"v1","addr":"172.21.15.87:6835","nonce":1831073338}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4398,"qdb_cluster":[4398]},"id":1}]} 2024-09-17T03:14:27.719 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 6 2024-09-17T03:14:27.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.722+0000 7f20c7e9b640 1 -- 172.21.15.87:0/3622389430 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f20a4078bc0 msgr2=0x7f20a407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:27.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.722+0000 7f20c7e9b640 1 --2- 172.21.15.87:0/3622389430 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f20a4078bc0 0x7f20a407b0d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f20c01ef8a0 tx=0x7f20b00079b0 comp rx=0 tx=0).stop 2024-09-17T03:14:27.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.722+0000 7f20c7e9b640 1 -- 172.21.15.87:0/3622389430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20c015c2d0 msgr2=0x7f20c00aadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:27.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.722+0000 7f20c7e9b640 1 --2- 172.21.15.87:0/3622389430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20c015c2d0 0x7f20c00aadf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00d7a0 tx=0x7f20bc00dc60 comp rx=0 tx=0).stop 2024-09-17T03:14:27.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.722+0000 7f20c7e9b640 1 -- 172.21.15.87:0/3622389430 shutdown_connections 2024-09-17T03:14:27.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.722+0000 7f20c7e9b640 1 --2- 172.21.15.87:0/3622389430 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f20a4078bc0 0x7f20a407b0d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:27.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.722+0000 7f20c7e9b640 1 --2- 172.21.15.87:0/3622389430 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f20c0150ac0 0x7f20c01f88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:27.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.722+0000 7f20c7e9b640 1 --2- 172.21.15.87:0/3622389430 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f20c015ecf0 0x7f20c015ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:27.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.722+0000 7f20c7e9b640 1 --2- 172.21.15.87:0/3622389430 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20c015c2d0 0x7f20c00aadf0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:27.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.722+0000 7f20c7e9b640 1 -- 172.21.15.87:0/3622389430 >> 172.21.15.87:0/3622389430 conn(0x7f20c00d4470 msgr2=0x7f20c00d4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:27.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.722+0000 7f20c7e9b640 1 -- 172.21.15.87:0/3622389430 shutdown_connections 2024-09-17T03:14:27.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.722+0000 7f20c7e9b640 1 -- 172.21.15.87:0/3622389430 wait complete. 2024-09-17T03:14:27.732 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:14:27.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.844+0000 7ff2f86cc640 1 -- 172.21.15.87:0/3827350054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2f00cbe70 msgr2=0x7ff2f015a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:27.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.844+0000 7ff2f86cc640 1 --2- 172.21.15.87:0/3827350054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2f00cbe70 0x7ff2f015a2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8005fd0 tx=0x7ff2e802a8b0 comp rx=0 tx=0).stop 2024-09-17T03:14:27.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.844+0000 7ff2f86cc640 1 -- 172.21.15.87:0/3827350054 shutdown_connections 2024-09-17T03:14:27.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.844+0000 7ff2f86cc640 1 --2- 172.21.15.87:0/3827350054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2f00cbe70 0x7ff2f015a2f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:27.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.844+0000 7ff2f86cc640 1 -- 172.21.15.87:0/3827350054 >> 172.21.15.87:0/3827350054 conn(0x7ff2f00d1110 msgr2=0x7ff2f00d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:27.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.844+0000 7ff2f86cc640 1 -- 172.21.15.87:0/3827350054 shutdown_connections 2024-09-17T03:14:27.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.844+0000 7ff2f86cc640 1 -- 172.21.15.87:0/3827350054 wait complete. 2024-09-17T03:14:27.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.846+0000 7ff2f86cc640 1 Processor -- start 2024-09-17T03:14:27.845 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.846+0000 7ff2f86cc640 1 -- start start 2024-09-17T03:14:27.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.847+0000 7ff2f86cc640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2f00b03d0 0x7ff2f01524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:27.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.847+0000 7ff2f86cc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f01602e0 0x7ff2f00b3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:27.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.847+0000 7ff2f86cc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2f0157130 0x7ff2f00b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:27.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.847+0000 7ff2f86cc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f015d5d0 con 0x7ff2f0157130 2024-09-17T03:14:27.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.847+0000 7ff2f86cc640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f015d450 con 0x7ff2f01602e0 2024-09-17T03:14:27.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.847+0000 7ff2f86cc640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff2f015d750 con 0x7ff2f00b03d0 2024-09-17T03:14:27.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.847+0000 7ff2f6c69640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2f0157130 0x7ff2f00b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:27.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.847+0000 7ff2f6c69640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2f0157130 0x7ff2f00b3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50936/0 (socket says 172.21.15.87:50936) 2024-09-17T03:14:27.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.847+0000 7ff2f6c69640 1 -- 172.21.15.87:0/3615130956 learned_addr learned my addr 172.21.15.87:0/3615130956 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:27.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.848+0000 7ff2f6c69640 1 -- 172.21.15.87:0/3615130956 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2f00b03d0 msgr2=0x7ff2f01524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:27.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.848+0000 7ff2f5c67640 1 --2- 172.21.15.87:0/3615130956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f01602e0 0x7ff2f00b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:27.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.848+0000 7ff2f6c69640 1 --2- 172.21.15.87:0/3615130956 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2f00b03d0 0x7ff2f01524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:27.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.848+0000 7ff2f6c69640 1 -- 172.21.15.87:0/3615130956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f01602e0 msgr2=0x7ff2f00b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:27.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.848+0000 7ff2f6c69640 1 --2- 172.21.15.87:0/3615130956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f01602e0 0x7ff2f00b3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:27.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.848+0000 7ff2f6c69640 1 -- 172.21.15.87:0/3615130956 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f01f3190 con 0x7ff2f0157130 2024-09-17T03:14:27.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.848+0000 7ff2f5c67640 1 --2- 172.21.15.87:0/3615130956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f01602e0 0x7ff2f00b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:14:27.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.848+0000 7ff2f6c69640 1 --2- 172.21.15.87:0/3615130956 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2f0157130 0x7ff2f00b3570 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff2e802aea0 tx=0x7ff2e80316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:27.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.848+0000 7ff2eeffd640 1 -- 172.21.15.87:0/3615130956 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8031990 con 0x7ff2f0157130 2024-09-17T03:14:27.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.848+0000 7ff2eeffd640 1 -- 172.21.15.87:0/3615130956 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2e8031c20 con 0x7ff2f0157130 2024-09-17T03:14:27.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.848+0000 7ff2eeffd640 1 -- 172.21.15.87:0/3615130956 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e80330d0 con 0x7ff2f0157130 2024-09-17T03:14:27.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.848+0000 7ff2f86cc640 1 -- 172.21.15.87:0/3615130956 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f01f3470 con 0x7ff2f0157130 2024-09-17T03:14:27.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.848+0000 7ff2f86cc640 1 -- 172.21.15.87:0/3615130956 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f01f3a30 con 0x7ff2f0157130 2024-09-17T03:14:27.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.849+0000 7ff2f86cc640 1 -- 172.21.15.87:0/3615130956 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2f015dbb0 con 0x7ff2f0157130 2024-09-17T03:14:27.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.850+0000 7ff2eeffd640 1 -- 172.21.15.87:0/3615130956 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff2e8033270 con 0x7ff2f0157130 2024-09-17T03:14:27.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.850+0000 7ff2eeffd640 1 --2- 172.21.15.87:0/3615130956 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff2dc078b80 0x7ff2dc07b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:27.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.851+0000 7ff2eeffd640 1 -- 172.21.15.87:0/3615130956 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff2e80bc6e0 con 0x7ff2f0157130 2024-09-17T03:14:27.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.851+0000 7ff2f6468640 1 --2- 172.21.15.87:0/3615130956 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff2dc078b80 0x7ff2dc07b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:27.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.851+0000 7ff2f6468640 1 --2- 172.21.15.87:0/3615130956 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff2dc078b80 0x7ff2dc07b090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0007fd0 tx=0x7ff2e0007900 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:27.852 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:27.854+0000 7ff2eeffd640 1 -- 172.21.15.87:0/3615130956 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff2e8052080 con 0x7ff2f0157130 2024-09-17T03:14:28.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.063+0000 7ff2f86cc640 1 -- 172.21.15.87:0/3615130956 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2f00b7090 con 0x7ff2f0157130 2024-09-17T03:14:28.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.063+0000 7ff2eeffd640 1 -- 172.21.15.87:0/3615130956 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5217 (secure 0 0 0) 0x7ff2e803f030 con 0x7ff2f0157130 2024-09-17T03:14:28.062 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:28.062 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":6,"btime":"2024-09-17T03:14:16:330267+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4289,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/110106418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":110106418},{"type":"v1","addr":"172.21.15.149:6835","nonce":110106418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1336626873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1336626873},{"type":"v1","addr":"172.21.15.149:6837","nonce":1336626873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/666860906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":666860906},{"type":"v1","addr":"172.21.15.87:6833","nonce":666860906}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:14:13.672621+0000","modified":"2024-09-17T03:14:15.675568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4398":{"gid":4398,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/1831073338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1831073338},{"type":"v1","addr":"172.21.15.87:6835","nonce":1831073338}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4398,"qdb_cluster":[4398]},"id":1}]} 2024-09-17T03:14:28.063 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 6 2024-09-17T03:14:28.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.065+0000 7ff2f86cc640 1 -- 172.21.15.87:0/3615130956 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff2dc078b80 msgr2=0x7ff2dc07b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:28.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.065+0000 7ff2f86cc640 1 --2- 172.21.15.87:0/3615130956 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff2dc078b80 0x7ff2dc07b090 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0007fd0 tx=0x7ff2e0007900 comp rx=0 tx=0).stop 2024-09-17T03:14:28.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.065+0000 7ff2f86cc640 1 -- 172.21.15.87:0/3615130956 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2f0157130 msgr2=0x7ff2f00b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:28.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.065+0000 7ff2f86cc640 1 --2- 172.21.15.87:0/3615130956 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2f0157130 0x7ff2f00b3570 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff2e802aea0 tx=0x7ff2e80316d0 comp rx=0 tx=0).stop 2024-09-17T03:14:28.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.065+0000 7ff2f86cc640 1 -- 172.21.15.87:0/3615130956 shutdown_connections 2024-09-17T03:14:28.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.065+0000 7ff2f86cc640 1 --2- 172.21.15.87:0/3615130956 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff2dc078b80 0x7ff2dc07b090 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:28.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.065+0000 7ff2f86cc640 1 --2- 172.21.15.87:0/3615130956 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff2f0157130 0x7ff2f00b3570 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:28.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.065+0000 7ff2f86cc640 1 --2- 172.21.15.87:0/3615130956 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff2f01602e0 0x7ff2f00b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:28.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.065+0000 7ff2f86cc640 1 --2- 172.21.15.87:0/3615130956 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff2f00b03d0 0x7ff2f01524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:28.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.066+0000 7ff2f86cc640 1 -- 172.21.15.87:0/3615130956 >> 172.21.15.87:0/3615130956 conn(0x7ff2f00d1110 msgr2=0x7ff2f0074330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:28.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.066+0000 7ff2f86cc640 1 -- 172.21.15.87:0/3615130956 shutdown_connections 2024-09-17T03:14:28.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.066+0000 7ff2f86cc640 1 -- 172.21.15.87:0/3615130956 wait complete. 2024-09-17T03:14:28.076 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:14:28.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.189+0000 7fcf3d721640 1 -- 172.21.15.87:0/392976185 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf381602e0 msgr2=0x7fcf3815a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:28.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.189+0000 7fcf3d721640 1 --2- 172.21.15.87:0/392976185 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf381602e0 0x7fcf3815a2f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009830 tx=0x7fcf2802a510 comp rx=0 tx=0).stop 2024-09-17T03:14:28.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.190+0000 7fcf3d721640 1 -- 172.21.15.87:0/392976185 shutdown_connections 2024-09-17T03:14:28.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.190+0000 7fcf3d721640 1 --2- 172.21.15.87:0/392976185 >> v2:172.21.15.149:3300/0 conn(0x7fcf380b03d0 0x7fcf380cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:28.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.190+0000 7fcf3d721640 1 --2- 172.21.15.87:0/392976185 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf381602e0 0x7fcf3815a2f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:28.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.190+0000 7fcf3d721640 1 -- 172.21.15.87:0/392976185 >> 172.21.15.87:0/392976185 conn(0x7fcf380d1110 msgr2=0x7fcf380d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:28.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.190+0000 7fcf3d721640 1 -- 172.21.15.87:0/392976185 shutdown_connections 2024-09-17T03:14:28.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.190+0000 7fcf3d721640 1 -- 172.21.15.87:0/392976185 wait complete. 2024-09-17T03:14:28.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.191+0000 7fcf3d721640 1 Processor -- start 2024-09-17T03:14:28.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.192+0000 7fcf3d721640 1 -- start start 2024-09-17T03:14:28.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.193+0000 7fcf3d721640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf380b03d0 0x7fcf381f6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:28.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.193+0000 7fcf3d721640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf380cb760 0x7fcf380b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:28.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.193+0000 7fcf3d721640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf381602e0 0x7fcf380b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:28.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.193+0000 7fcf3d721640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3815d5d0 con 0x7fcf381602e0 2024-09-17T03:14:28.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.193+0000 7fcf3d721640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3815d450 con 0x7fcf380cb760 2024-09-17T03:14:28.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.193+0000 7fcf3d721640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcf3815d750 con 0x7fcf380b03d0 2024-09-17T03:14:28.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.193+0000 7fcf377fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf381602e0 0x7fcf380b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:28.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.193+0000 7fcf377fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf381602e0 0x7fcf380b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50942/0 (socket says 172.21.15.87:50942) 2024-09-17T03:14:28.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.193+0000 7fcf377fe640 1 -- 172.21.15.87:0/2322400676 learned_addr learned my addr 172.21.15.87:0/2322400676 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:28.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.193+0000 7fcf367fc640 1 --2- 172.21.15.87:0/2322400676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf380cb760 0x7fcf380b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:28.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.193+0000 7fcf36ffd640 1 --2- 172.21.15.87:0/2322400676 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf380b03d0 0x7fcf381f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:28.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.193+0000 7fcf377fe640 1 -- 172.21.15.87:0/2322400676 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf380b03d0 msgr2=0x7fcf381f6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:28.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.193+0000 7fcf377fe640 1 --2- 172.21.15.87:0/2322400676 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf380b03d0 0x7fcf381f6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:28.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.193+0000 7fcf377fe640 1 -- 172.21.15.87:0/2322400676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf380cb760 msgr2=0x7fcf380b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:28.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.193+0000 7fcf377fe640 1 --2- 172.21.15.87:0/2322400676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf380cb760 0x7fcf380b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:28.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.193+0000 7fcf377fe640 1 -- 172.21.15.87:0/2322400676 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf380b1b00 con 0x7fcf381602e0 2024-09-17T03:14:28.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.193+0000 7fcf36ffd640 1 --2- 172.21.15.87:0/2322400676 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf380b03d0 0x7fcf381f6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:28.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.193+0000 7fcf367fc640 1 --2- 172.21.15.87:0/2322400676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf380cb760 0x7fcf380b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:28.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.193+0000 7fcf377fe640 1 --2- 172.21.15.87:0/2322400676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf381602e0 0x7fcf380b2e30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000c760 tx=0x7fcf3000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:28.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.194+0000 7fcf2f7fe640 1 -- 172.21.15.87:0/2322400676 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf30003b70 con 0x7fcf381602e0 2024-09-17T03:14:28.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.194+0000 7fcf3d721640 1 -- 172.21.15.87:0/2322400676 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf380b3fe0 con 0x7fcf381602e0 2024-09-17T03:14:28.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.194+0000 7fcf2f7fe640 1 -- 172.21.15.87:0/2322400676 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf30003d10 con 0x7fcf381602e0 2024-09-17T03:14:28.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.194+0000 7fcf2f7fe640 1 -- 172.21.15.87:0/2322400676 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf3000aab0 con 0x7fcf381602e0 2024-09-17T03:14:28.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.194+0000 7fcf3d721640 1 -- 172.21.15.87:0/2322400676 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf380b45a0 con 0x7fcf381602e0 2024-09-17T03:14:28.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.195+0000 7fcf3d721640 1 -- 172.21.15.87:0/2322400676 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf3815e3c0 con 0x7fcf381602e0 2024-09-17T03:14:28.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.196+0000 7fcf2f7fe640 1 -- 172.21.15.87:0/2322400676 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcf3000ac50 con 0x7fcf381602e0 2024-09-17T03:14:28.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.196+0000 7fcf2f7fe640 1 --2- 172.21.15.87:0/2322400676 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcf18078bc0 0x7fcf1807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:28.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.196+0000 7fcf2f7fe640 1 -- 172.21.15.87:0/2322400676 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fcf3009aa80 con 0x7fcf381602e0 2024-09-17T03:14:28.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.196+0000 7fcf36ffd640 1 --2- 172.21.15.87:0/2322400676 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcf18078bc0 0x7fcf1807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:28.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.197+0000 7fcf36ffd640 1 --2- 172.21.15.87:0/2322400676 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcf18078bc0 0x7fcf1807b0d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcf28034000 tx=0x7fcf28004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:28.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.200+0000 7fcf2f7fe640 1 -- 172.21.15.87:0/2322400676 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fcf30062ed0 con 0x7fcf381602e0 2024-09-17T03:14:28.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.376+0000 7fcf3d721640 1 -- 172.21.15.87:0/2322400676 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf380d0cb0 con 0x7fcf381602e0 2024-09-17T03:14:28.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.377+0000 7fcf2f7fe640 1 -- 172.21.15.87:0/2322400676 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16693 (secure 0 0 0) 0x7fcf300627d0 con 0x7fcf381602e0 2024-09-17T03:14:28.376 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:28.376 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":23,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:14:15.323716+0000","last_up_change":"2024-09-17T03:14:01.833605+0000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:14:03.433404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:14:11.525547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T03:14:12.626396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6825","nonce":2430409749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6827","nonce":2430409749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6831","nonce":2430409749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6829","nonce":2430409749}]},"public_addr":"172.21.15.87:6825/2430409749","cluster_addr":"172.21.15.87:6827/2430409749","heartbeat_back_addr":"172.21.15.87:6831/2430409749","heartbeat_front_addr":"172.21.15.87:6829/2430409749","state":["exists","up"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6809","nonce":3467605435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6811","nonce":3467605435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6815","nonce":3467605435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6813","nonce":3467605435}]},"public_addr":"172.21.15.87:6809/3467605435","cluster_addr":"172.21.15.87:6811/3467605435","heartbeat_back_addr":"172.21.15.87:6815/3467605435","heartbeat_front_addr":"172.21.15.87:6813/3467605435","state":["exists","up"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6801","nonce":1450834766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6803","nonce":1450834766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6807","nonce":1450834766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6805","nonce":1450834766}]},"public_addr":"172.21.15.87:6801/1450834766","cluster_addr":"172.21.15.87:6803/1450834766","heartbeat_back_addr":"172.21.15.87:6807/1450834766","heartbeat_front_addr":"172.21.15.87:6805/1450834766","state":["exists","up"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6817","nonce":4253161216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6819","nonce":4253161216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6823","nonce":4253161216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6821","nonce":4253161216}]},"public_addr":"172.21.15.87:6817/4253161216","cluster_addr":"172.21.15.87:6819/4253161216","heartbeat_back_addr":"172.21.15.87:6823/4253161216","heartbeat_front_addr":"172.21.15.87:6821/4253161216","state":["exists","up"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6803","nonce":2414907089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6805","nonce":2414907089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6809","nonce":2414907089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6807","nonce":2414907089}]},"public_addr":"172.21.15.149:6803/2414907089","cluster_addr":"172.21.15.149:6805/2414907089","heartbeat_back_addr":"172.21.15.149:6809/2414907089","heartbeat_front_addr":"172.21.15.149:6807/2414907089","state":["exists","up"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6827","nonce":2182999530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6829","nonce":2182999530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6833","nonce":2182999530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6831","nonce":2182999530}]},"public_addr":"172.21.15.149:6827/2182999530","cluster_addr":"172.21.15.149:6829/2182999530","heartbeat_back_addr":"172.21.15.149:6833/2182999530","heartbeat_front_addr":"172.21.15.149:6831/2182999530","state":["exists","up"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6811","nonce":4289519146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6813","nonce":4289519146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6817","nonce":4289519146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6815","nonce":4289519146}]},"public_addr":"172.21.15.149:6811/4289519146","cluster_addr":"172.21.15.149:6813/4289519146","heartbeat_back_addr":"172.21.15.149:6817/4289519146","heartbeat_front_addr":"172.21.15.149:6815/4289519146","state":["exists","up"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6819","nonce":2771010477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6821","nonce":2771010477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6825","nonce":2771010477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6823","nonce":2771010477}]},"public_addr":"172.21.15.149:6819/2771010477","cluster_addr":"172.21.15.149:6821/2771010477","heartbeat_back_addr":"172.21.15.149:6825/2771010477","heartbeat_front_addr":"172.21.15.149:6823/2771010477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.313297+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.642861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.654781+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.302606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.385791+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.915266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.121365+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.394432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:14:28.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.379+0000 7fcf3d721640 1 -- 172.21.15.87:0/2322400676 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcf18078bc0 msgr2=0x7fcf1807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:28.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.379+0000 7fcf3d721640 1 --2- 172.21.15.87:0/2322400676 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcf18078bc0 0x7fcf1807b0d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fcf28034000 tx=0x7fcf28004730 comp rx=0 tx=0).stop 2024-09-17T03:14:28.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.379+0000 7fcf3d721640 1 -- 172.21.15.87:0/2322400676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf381602e0 msgr2=0x7fcf380b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:28.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.379+0000 7fcf3d721640 1 --2- 172.21.15.87:0/2322400676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf381602e0 0x7fcf380b2e30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000c760 tx=0x7fcf3000cc20 comp rx=0 tx=0).stop 2024-09-17T03:14:28.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.379+0000 7fcf3d721640 1 -- 172.21.15.87:0/2322400676 shutdown_connections 2024-09-17T03:14:28.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.379+0000 7fcf3d721640 1 --2- 172.21.15.87:0/2322400676 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcf18078bc0 0x7fcf1807b0d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:28.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.379+0000 7fcf3d721640 1 --2- 172.21.15.87:0/2322400676 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf381602e0 0x7fcf380b2e30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:28.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.379+0000 7fcf3d721640 1 --2- 172.21.15.87:0/2322400676 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf380cb760 0x7fcf380b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:28.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.379+0000 7fcf3d721640 1 --2- 172.21.15.87:0/2322400676 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf380b03d0 0x7fcf381f6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:28.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.379+0000 7fcf3d721640 1 -- 172.21.15.87:0/2322400676 >> 172.21.15.87:0/2322400676 conn(0x7fcf380d1110 msgr2=0x7fcf380d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:28.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.379+0000 7fcf3d721640 1 -- 172.21.15.87:0/2322400676 shutdown_connections 2024-09-17T03:14:28.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.379+0000 7fcf3d721640 1 -- 172.21.15.87:0/2322400676 wait complete. 2024-09-17T03:14:28.389 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:14:28.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.501+0000 7f120acd3640 1 -- 172.21.15.87:0/4137932088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12040afac0 msgr2=0x7f120415af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:28.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.501+0000 7f120acd3640 1 --2- 172.21.15.87:0/4137932088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12040afac0 0x7f120415af00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f11f8005fd0 tx=0x7f11f802b910 comp rx=0 tx=0).stop 2024-09-17T03:14:28.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.501+0000 7f120acd3640 1 -- 172.21.15.87:0/4137932088 shutdown_connections 2024-09-17T03:14:28.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.501+0000 7f120acd3640 1 --2- 172.21.15.87:0/4137932088 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1204073460 0x7f1204073870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:28.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.501+0000 7f120acd3640 1 --2- 172.21.15.87:0/4137932088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12040afac0 0x7f120415af00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:28.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.501+0000 7f120acd3640 1 -- 172.21.15.87:0/4137932088 >> 172.21.15.87:0/4137932088 conn(0x7f12040d08c0 msgr2=0x7f12040d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:28.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.502+0000 7f120acd3640 1 -- 172.21.15.87:0/4137932088 shutdown_connections 2024-09-17T03:14:28.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.502+0000 7f120acd3640 1 -- 172.21.15.87:0/4137932088 wait complete. 2024-09-17T03:14:28.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.503+0000 7f120acd3640 1 Processor -- start 2024-09-17T03:14:28.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.503+0000 7f120acd3640 1 -- start start 2024-09-17T03:14:28.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.505+0000 7f120acd3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1204073460 0x7f12040aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:28.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.505+0000 7f120acd3640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f12040afac0 0x7f1204157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:28.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.505+0000 7f120acd3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12040cc130 0x7f12041e6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:28.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.505+0000 7f120acd3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1204069110 con 0x7f12040cc130 2024-09-17T03:14:28.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.505+0000 7f120acd3640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1204068f90 con 0x7f1204073460 2024-09-17T03:14:28.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.505+0000 7f120acd3640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1204069290 con 0x7f12040afac0 2024-09-17T03:14:28.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.505+0000 7f1209270640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12040cc130 0x7f12041e6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:28.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.505+0000 7f1209270640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12040cc130 0x7f12041e6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50960/0 (socket says 172.21.15.87:50960) 2024-09-17T03:14:28.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.505+0000 7f1209270640 1 -- 172.21.15.87:0/2808029484 learned_addr learned my addr 172.21.15.87:0/2808029484 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:28.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.505+0000 7f1203fff640 1 --2- 172.21.15.87:0/2808029484 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f12040afac0 0x7f1204157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:28.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.505+0000 7f1208a6f640 1 --2- 172.21.15.87:0/2808029484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1204073460 0x7f12040aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:28.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.505+0000 7f1209270640 1 -- 172.21.15.87:0/2808029484 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f12040afac0 msgr2=0x7f1204157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:28.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.505+0000 7f1209270640 1 --2- 172.21.15.87:0/2808029484 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f12040afac0 0x7f1204157a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:28.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.505+0000 7f1209270640 1 -- 172.21.15.87:0/2808029484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1204073460 msgr2=0x7f12040aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:28.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.505+0000 7f1209270640 1 --2- 172.21.15.87:0/2808029484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1204073460 0x7f12040aee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:28.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.505+0000 7f1209270640 1 -- 172.21.15.87:0/2808029484 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12040c7bc0 con 0x7f12040cc130 2024-09-17T03:14:28.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.505+0000 7f1209270640 1 --2- 172.21.15.87:0/2808029484 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12040cc130 0x7f12041e6990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00ea40 tx=0x7f11fc00ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:28.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.505+0000 7f12017fa640 1 -- 172.21.15.87:0/2808029484 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc00ce40 con 0x7f12040cc130 2024-09-17T03:14:28.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.505+0000 7f12017fa640 1 -- 172.21.15.87:0/2808029484 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11fc004c10 con 0x7f12040cc130 2024-09-17T03:14:28.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.505+0000 7f120acd3640 1 -- 172.21.15.87:0/2808029484 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12040c7ea0 con 0x7f12040cc130 2024-09-17T03:14:28.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.506+0000 7f12017fa640 1 -- 172.21.15.87:0/2808029484 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc005020 con 0x7f12040cc130 2024-09-17T03:14:28.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.506+0000 7f120acd3640 1 -- 172.21.15.87:0/2808029484 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1204158180 con 0x7f12040cc130 2024-09-17T03:14:28.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.507+0000 7f120acd3640 1 -- 172.21.15.87:0/2808029484 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12040708b0 con 0x7f12040cc130 2024-09-17T03:14:28.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.507+0000 7f12017fa640 1 -- 172.21.15.87:0/2808029484 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f11fc0051c0 con 0x7f12040cc130 2024-09-17T03:14:28.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.508+0000 7f12017fa640 1 --2- 172.21.15.87:0/2808029484 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f11e8078bc0 0x7f11e807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:28.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.508+0000 7f12017fa640 1 -- 172.21.15.87:0/2808029484 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f11fc09b990 con 0x7f12040cc130 2024-09-17T03:14:28.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.508+0000 7f1208a6f640 1 --2- 172.21.15.87:0/2808029484 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f11e8078bc0 0x7f11e807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:28.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.509+0000 7f1208a6f640 1 --2- 172.21.15.87:0/2808029484 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f11e8078bc0 0x7f11e807b0d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f11f8035000 tx=0x7f11f8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:28.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.512+0000 7f12017fa640 1 -- 172.21.15.87:0/2808029484 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f12040708b0 con 0x7f12040cc130 2024-09-17T03:14:28.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.721+0000 7f120acd3640 1 -- 172.21.15.87:0/2808029484 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12040af4e0 con 0x7f12040cc130 2024-09-17T03:14:28.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.722+0000 7f12017fa640 1 -- 172.21.15.87:0/2808029484 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5217 (secure 0 0 0) 0x7f11fc063810 con 0x7f12040cc130 2024-09-17T03:14:28.721 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:28.721 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":6,"btime":"2024-09-17T03:14:16:330267+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4289,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/110106418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":110106418},{"type":"v1","addr":"172.21.15.149:6835","nonce":110106418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1336626873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1336626873},{"type":"v1","addr":"172.21.15.149:6837","nonce":1336626873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/666860906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":666860906},{"type":"v1","addr":"172.21.15.87:6833","nonce":666860906}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:14:13.672621+0000","modified":"2024-09-17T03:14:15.675568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4398":{"gid":4398,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/1831073338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1831073338},{"type":"v1","addr":"172.21.15.87:6835","nonce":1831073338}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4398,"qdb_cluster":[4398]},"id":1}]} 2024-09-17T03:14:28.721 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 6 2024-09-17T03:14:28.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.724+0000 7f120acd3640 1 -- 172.21.15.87:0/2808029484 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f11e8078bc0 msgr2=0x7f11e807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:28.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.724+0000 7f120acd3640 1 --2- 172.21.15.87:0/2808029484 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f11e8078bc0 0x7f11e807b0d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f11f8035000 tx=0x7f11f8004730 comp rx=0 tx=0).stop 2024-09-17T03:14:28.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.724+0000 7f120acd3640 1 -- 172.21.15.87:0/2808029484 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12040cc130 msgr2=0x7f12041e6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:28.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.724+0000 7f120acd3640 1 --2- 172.21.15.87:0/2808029484 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12040cc130 0x7f12041e6990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00ea40 tx=0x7f11fc00ef00 comp rx=0 tx=0).stop 2024-09-17T03:14:28.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.724+0000 7f120acd3640 1 -- 172.21.15.87:0/2808029484 shutdown_connections 2024-09-17T03:14:28.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.724+0000 7f120acd3640 1 --2- 172.21.15.87:0/2808029484 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f11e8078bc0 0x7f11e807b0d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:28.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.724+0000 7f120acd3640 1 --2- 172.21.15.87:0/2808029484 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12040cc130 0x7f12041e6990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:28.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.724+0000 7f120acd3640 1 --2- 172.21.15.87:0/2808029484 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f12040afac0 0x7f1204157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:28.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.724+0000 7f120acd3640 1 --2- 172.21.15.87:0/2808029484 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1204073460 0x7f12040aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:28.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.724+0000 7f120acd3640 1 -- 172.21.15.87:0/2808029484 >> 172.21.15.87:0/2808029484 conn(0x7f12040d08c0 msgr2=0x7f120415a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:28.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.724+0000 7f120acd3640 1 -- 172.21.15.87:0/2808029484 shutdown_connections 2024-09-17T03:14:28.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:28.724+0000 7f120acd3640 1 -- 172.21.15.87:0/2808029484 wait complete. 2024-09-17T03:14:29.138 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T03:14:13.672621+0000', 'modified': '2024-09-17T03:14:15.675568+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4398}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4398': {'gid': 4398, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.87:6835/1831073338', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6834', 'nonce': 1831073338}, {'type': 'v1', 'addr': '172.21.15.87:6835', 'nonce': 1831073338}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4398, 'qdb_cluster': [4398]} 2024-09-17T03:14:29.139 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T03:14:29.139 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-17T03:14:29.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.259+0000 7f5781982640 1 -- 172.21.15.87:0/3567562735 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f577c0d3060 msgr2=0x7f577c0cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:29.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.259+0000 7f5781982640 1 --2- 172.21.15.87:0/3567562735 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f577c0d3060 0x7f577c0cf6c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5774005fd0 tx=0x7f577402a8b0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.259+0000 7f5781982640 1 -- 172.21.15.87:0/3567562735 shutdown_connections 2024-09-17T03:14:29.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.259+0000 7f5781982640 1 --2- 172.21.15.87:0/3567562735 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f577c0d3060 0x7f577c0cf6c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.259+0000 7f5781982640 1 -- 172.21.15.87:0/3567562735 >> 172.21.15.87:0/3567562735 conn(0x7f577c08dae0 msgr2=0x7f577c09d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:29.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.259+0000 7f5781982640 1 -- 172.21.15.87:0/3567562735 shutdown_connections 2024-09-17T03:14:29.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.259+0000 7f5781982640 1 -- 172.21.15.87:0/3567562735 wait complete. 2024-09-17T03:14:29.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.261+0000 7f5781982640 1 Processor -- start 2024-09-17T03:14:29.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.261+0000 7f5781982640 1 -- start start 2024-09-17T03:14:29.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.263+0000 7f5781982640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f577c0d2c50 0x7f577c09e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:29.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.263+0000 7f5781982640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f577c0d3060 0x7f577c0d1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:29.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.263+0000 7f5781982640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f577c0d4860 0x7f577c15e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:29.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.263+0000 7f5781982640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f577c0cfd80 con 0x7f577c0d4860 2024-09-17T03:14:29.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.263+0000 7f5781982640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f577c0cfc00 con 0x7f577c0d3060 2024-09-17T03:14:29.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.263+0000 7f5781982640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f577c0cff00 con 0x7f577c0d2c50 2024-09-17T03:14:29.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.263+0000 7f577b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f577c0d4860 0x7f577c15e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:29.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.263+0000 7f577b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f577c0d4860 0x7f577c15e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50968/0 (socket says 172.21.15.87:50968) 2024-09-17T03:14:29.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.263+0000 7f577b7fe640 1 -- 172.21.15.87:0/1114566348 learned_addr learned my addr 172.21.15.87:0/1114566348 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:29.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.263+0000 7f577affd640 1 --2- 172.21.15.87:0/1114566348 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f577c0d2c50 0x7f577c09e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:29.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.263+0000 7f57727fc640 1 --2- 172.21.15.87:0/1114566348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f577c0d3060 0x7f577c0d1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:29.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.263+0000 7f577b7fe640 1 -- 172.21.15.87:0/1114566348 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f577c0d2c50 msgr2=0x7f577c09e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:29.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.263+0000 7f577b7fe640 1 --2- 172.21.15.87:0/1114566348 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f577c0d2c50 0x7f577c09e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.263+0000 7f577b7fe640 1 -- 172.21.15.87:0/1114566348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f577c0d3060 msgr2=0x7f577c0d1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:29.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.263+0000 7f577b7fe640 1 --2- 172.21.15.87:0/1114566348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f577c0d3060 0x7f577c0d1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.263+0000 7f577b7fe640 1 -- 172.21.15.87:0/1114566348 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577c0a2340 con 0x7f577c0d4860 2024-09-17T03:14:29.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.263+0000 7f577b7fe640 1 --2- 172.21.15.87:0/1114566348 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f577c0d4860 0x7f577c15e8c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5774006010 tx=0x7f5774003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:29.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.263+0000 7f5778ff9640 1 -- 172.21.15.87:0/1114566348 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774031b90 con 0x7f577c0d4860 2024-09-17T03:14:29.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.263+0000 7f5778ff9640 1 -- 172.21.15.87:0/1114566348 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5774031e20 con 0x7f577c0d4860 2024-09-17T03:14:29.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.263+0000 7f5781982640 1 -- 172.21.15.87:0/1114566348 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f577c1f5dc0 con 0x7f577c0d4860 2024-09-17T03:14:29.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.263+0000 7f5778ff9640 1 -- 172.21.15.87:0/1114566348 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774033310 con 0x7f577c0d4860 2024-09-17T03:14:29.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.263+0000 7f5781982640 1 -- 172.21.15.87:0/1114566348 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f577c1f6380 con 0x7f577c0d4860 2024-09-17T03:14:29.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.266+0000 7f5778ff9640 1 -- 172.21.15.87:0/1114566348 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5774031660 con 0x7f577c0d4860 2024-09-17T03:14:29.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.266+0000 7f5778ff9640 1 --2- 172.21.15.87:0/1114566348 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f575c078bc0 0x7f575c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:29.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.266+0000 7f5778ff9640 1 -- 172.21.15.87:0/1114566348 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f57740bc3d0 con 0x7f577c0d4860 2024-09-17T03:14:29.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.266+0000 7f577affd640 1 --2- 172.21.15.87:0/1114566348 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f575c078bc0 0x7f575c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:29.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.267+0000 7f57889cb740 1 Processor -- start 2024-09-17T03:14:29.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.267+0000 7f57889cb740 1 -- start start 2024-09-17T03:14:29.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.267+0000 7f577affd640 1 --2- 172.21.15.87:0/1114566348 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f575c078bc0 0x7f575c07b0d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f576c0046b0 tx=0x7f576c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:29.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.268+0000 7f57889cb740 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55f9d5661270 0x55f9d5a32fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:29.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.268+0000 7f57889cb740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d56383b0 0x55f9d5a334e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:29.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.268+0000 7f57889cb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9d5640590 0x55f9d5a503e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:29.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.268+0000 7f57889cb740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f9d597b820 con 0x55f9d56383b0 2024-09-17T03:14:29.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.268+0000 7f57889cb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f9d596ac50 con 0x55f9d5640590 2024-09-17T03:14:29.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.268+0000 7f57889cb740 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55f9d596ea60 con 0x55f9d5661270 2024-09-17T03:14:29.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.268+0000 7f57727fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d56383b0 0x55f9d5a334e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:29.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.268+0000 7f57727fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d56383b0 0x55f9d5a334e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50974/0 (socket says 172.21.15.87:50974) 2024-09-17T03:14:29.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.268+0000 7f57727fc640 1 -- 172.21.15.87:0/1382698803 learned_addr learned my addr 172.21.15.87:0/1382698803 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:29.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.269+0000 7f577affd640 1 --2- 172.21.15.87:0/1382698803 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55f9d5661270 0x55f9d5a32fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:29.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.269+0000 7f577b7fe640 1 --2- 172.21.15.87:0/1382698803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9d5640590 0x55f9d5a503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:29.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.269+0000 7f57727fc640 1 -- 172.21.15.87:0/1382698803 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55f9d5661270 msgr2=0x55f9d5a32fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:14:29.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.269+0000 7f57727fc640 1 --2- 172.21.15.87:0/1382698803 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55f9d5661270 0x55f9d5a32fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.269+0000 7f57727fc640 1 -- 172.21.15.87:0/1382698803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9d5640590 msgr2=0x55f9d5a503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:14:29.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.269+0000 7f57727fc640 1 --2- 172.21.15.87:0/1382698803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9d5640590 0x55f9d5a503e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.269+0000 7f57727fc640 1 -- 172.21.15.87:0/1382698803 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9d56cc850 con 0x55f9d56383b0 2024-09-17T03:14:29.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.269+0000 7f57727fc640 1 --2- 172.21.15.87:0/1382698803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d56383b0 0x55f9d5a334e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f57640014e0 tx=0x7f57640019a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3088a8e3f5ee32fd server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:29.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.269+0000 7f57717fa640 1 -- 172.21.15.87:0/1382698803 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764019070 con 0x55f9d56383b0 2024-09-17T03:14:29.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.269+0000 7f57717fa640 1 -- 172.21.15.87:0/1382698803 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57640052e0 con 0x55f9d56383b0 2024-09-17T03:14:29.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.269+0000 7f57717fa640 1 -- 172.21.15.87:0/1382698803 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5764003850 con 0x55f9d56383b0 2024-09-17T03:14:29.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.270+0000 7f57889cb740 1 -- 172.21.15.87:0/1382698803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d56383b0 msgr2=0x55f9d5a334e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:29.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.270+0000 7f57889cb740 1 --2- 172.21.15.87:0/1382698803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d56383b0 0x55f9d5a334e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f57640014e0 tx=0x7f57640019a0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.270+0000 7f57889cb740 1 -- 172.21.15.87:0/1382698803 shutdown_connections 2024-09-17T03:14:29.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.270+0000 7f57889cb740 1 --2- 172.21.15.87:0/1382698803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9d5640590 0x55f9d5a503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.270+0000 7f57889cb740 1 --2- 172.21.15.87:0/1382698803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d56383b0 0x55f9d5a334e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.270+0000 7f57889cb740 1 --2- 172.21.15.87:0/1382698803 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55f9d5661270 0x55f9d5a32fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.270+0000 7f57889cb740 1 -- 172.21.15.87:0/1382698803 >> 172.21.15.87:0/1382698803 conn(0x55f9d56553c0 msgr2=0x55f9d5a09260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:29.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.270+0000 7f57889cb740 1 -- 172.21.15.87:0/1382698803 shutdown_connections 2024-09-17T03:14:29.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.270+0000 7f57889cb740 1 -- 172.21.15.87:0/1382698803 wait complete. 2024-09-17T03:14:29.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.271+0000 7f57889cb740 1 -- start start 2024-09-17T03:14:29.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.271+0000 7f57889cb740 1 Processor -- start 2024-09-17T03:14:29.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.272+0000 7f57889cb740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d56383b0 0x55f9d5a462d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:29.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.272+0000 7f57889cb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9d5640590 0x55f9d5a44b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:29.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.272+0000 7f57889cb740 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55f9d5661270 0x55f9d5a60670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:29.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.272+0000 7f57889cb740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f9d56db620 con 0x55f9d56383b0 2024-09-17T03:14:29.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.272+0000 7f57889cb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f9d59105f0 con 0x55f9d5640590 2024-09-17T03:14:29.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.272+0000 7f57889cb740 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55f9d5a33a20 con 0x55f9d5661270 2024-09-17T03:14:29.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.272+0000 7f57889cb740 10 client.0 _ll_register_callbacks cb 0x55f9d59140a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:14:29.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.272+0000 7f577affd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d56383b0 0x55f9d5a462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:29.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.272+0000 7f577affd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d56383b0 0x55f9d5a462d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50986/0 (socket says 172.21.15.87:50986) 2024-09-17T03:14:29.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.272+0000 7f577affd640 1 -- 172.21.15.87:0/3830888470 learned_addr learned my addr 172.21.15.87:0/3830888470 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:29.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.272+0000 7f577b7fe640 1 --2- 172.21.15.87:0/3830888470 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55f9d5661270 0x55f9d5a60670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:29.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.272+0000 7f57727fc640 1 --2- 172.21.15.87:0/3830888470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9d5640590 0x55f9d5a44b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:29.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.273+0000 7f577affd640 1 -- 172.21.15.87:0/3830888470 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55f9d5661270 msgr2=0x55f9d5a60670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:14:29.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.273+0000 7f577affd640 1 --2- 172.21.15.87:0/3830888470 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55f9d5661270 0x55f9d5a60670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.273+0000 7f577affd640 1 -- 172.21.15.87:0/3830888470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9d5640590 msgr2=0x55f9d5a44b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:14:29.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.273+0000 7f577affd640 1 --2- 172.21.15.87:0/3830888470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9d5640590 0x55f9d5a44b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.273+0000 7f577affd640 1 -- 172.21.15.87:0/3830888470 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9d57436b0 con 0x55f9d56383b0 2024-09-17T03:14:29.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.273+0000 7f577affd640 1 --2- 172.21.15.87:0/3830888470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d56383b0 0x55f9d5a462d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f576c00ecd0 tx=0x7f576c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e212776cd03c02 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:29.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.273+0000 7f575a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-09-17T03:14:29.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.273+0000 7f575a7fc640 1 -- 172.21.15.87:0/3830888470 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c021420 con 0x55f9d56383b0 2024-09-17T03:14:29.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.273+0000 7f575a7fc640 1 -- 172.21.15.87:0/3830888470 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f576c010040 con 0x55f9d56383b0 2024-09-17T03:14:29.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.273+0000 7f57889cb740 1 -- 172.21.15.87:0/3830888470 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57640052e0 con 0x55f9d56383b0 2024-09-17T03:14:29.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.273+0000 7f575a7fc640 1 -- 172.21.15.87:0/3830888470 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f576c02e050 con 0x55f9d56383b0 2024-09-17T03:14:29.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.273+0000 7f57889cb740 10 client.4752 fetch_fsmap learned FSMap version 6 2024-09-17T03:14:29.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.273+0000 7f57889cb740 1 -- 172.21.15.87:0/3830888470 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55f9d5a2b2d0 con 0x55f9d56383b0 2024-09-17T03:14:29.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.273+0000 7f575a7fc640 1 -- 172.21.15.87:0/3830888470 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57640052e0 con 0x55f9d56383b0 2024-09-17T03:14:29.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.273+0000 7f575a7fc640 1 -- 172.21.15.87:0/3830888470 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 6) ==== 3502+0+0 (secure 0 0 0) 0x7f576c031020 con 0x55f9d56383b0 2024-09-17T03:14:29.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.273+0000 7f57889cb740 10 client.4752 fetch_fsmap finished waiting for FSMap version 6 2024-09-17T03:14:29.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.273+0000 7f57889cb740 10 client.4752 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T03:14:29.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.273+0000 7f57889cb740 20 client.4752 populate_metadata read hostname 'smithi087' 2024-09-17T03:14:29.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.273+0000 7f57889cb740 1 --2- 172.21.15.87:0/3830888470 >> [v2:172.21.15.87:6834/1831073338,v1:172.21.15.87:6835/1831073338] conn(0x55f9d561fe40 0x55f9d5a5d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:29.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.273+0000 7f57889cb740 4 client.4752 mds_command: new command op to 4398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T03:14:29.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.273+0000 7f57889cb740 1 -- 172.21.15.87:0/3830888470 --> [v2:172.21.15.87:6834/1831073338,v1:172.21.15.87:6835/1831073338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9d597d770 con 0x55f9d561fe40 2024-09-17T03:14:29.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.274+0000 7f57727fc640 1 --2- 172.21.15.87:0/3830888470 >> [v2:172.21.15.87:6834/1831073338,v1:172.21.15.87:6835/1831073338] conn(0x55f9d561fe40 0x55f9d5a5d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.274+0000 7f575a7fc640 10 client.4752 ms_handle_connect on v2:172.21.15.87:6834/1831073338 2024-09-17T03:14:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.274+0000 7f57727fc640 1 --2- 172.21.15.87:0/3830888470 >> [v2:172.21.15.87:6834/1831073338,v1:172.21.15.87:6835/1831073338] conn(0x55f9d561fe40 0x55f9d5a5d080 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5695733401284871 server_cookie=3cfa6727432bc9c4 in_seq=0 out_seq=0 2024-09-17T03:14:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.275+0000 7f575a7fc640 1 -- 172.21.15.87:0/3830888470 <== mds.0 v2:172.21.15.87:6834/1831073338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f9d597d770 con 0x55f9d561fe40 2024-09-17T03:14:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.275+0000 7f575a7fc640 10 client.4752 handle_command_reply: tid=0 2024-09-17T03:14:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.275+0000 7f57889cb740 1 client.4752 shutdown 2024-09-17T03:14:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.275+0000 7f57889cb740 1 -- 172.21.15.87:0/3830888470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d56383b0 msgr2=0x55f9d5a462d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.275+0000 7f57889cb740 1 --2- 172.21.15.87:0/3830888470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d56383b0 0x55f9d5a462d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f576c00ecd0 tx=0x7f576c000ed0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.275+0000 7f57889cb740 1 -- 172.21.15.87:0/3830888470 shutdown_connections 2024-09-17T03:14:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.275+0000 7f57889cb740 1 --2- 172.21.15.87:0/3830888470 >> [v2:172.21.15.87:6834/1831073338,v1:172.21.15.87:6835/1831073338] conn(0x55f9d561fe40 0x55f9d5a5d080 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.275+0000 7f57889cb740 1 --2- 172.21.15.87:0/3830888470 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55f9d5661270 0x55f9d5a60670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.275+0000 7f57889cb740 1 --2- 172.21.15.87:0/3830888470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9d5640590 0x55f9d5a44b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.275+0000 7f57889cb740 1 --2- 172.21.15.87:0/3830888470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d56383b0 0x55f9d5a462d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.275+0000 7f575a7fc640 0 client.4752 ms_handle_reset on v2:172.21.15.87:6834/1831073338 2024-09-17T03:14:29.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.275+0000 7f57889cb740 1 -- 172.21.15.87:0/3830888470 >> 172.21.15.87:0/3830888470 conn(0x55f9d56553c0 msgr2=0x55f9d5a51af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:29.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.275+0000 7f57889cb740 1 -- 172.21.15.87:0/3830888470 shutdown_connections 2024-09-17T03:14:29.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.275+0000 7f57889cb740 1 -- 172.21.15.87:0/3830888470 wait complete. 2024-09-17T03:14:29.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.275+0000 7f57889cb740 20 client.4752 trim_cache size 0 max 16384 2024-09-17T03:14:29.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.293+0000 7f57889cb740 1 Processor -- start 2024-09-17T03:14:29.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.293+0000 7f57889cb740 1 -- start start 2024-09-17T03:14:29.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.294+0000 7f57889cb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9d56383b0 0x55f9d5a08460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:29.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.294+0000 7f57889cb740 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55f9d5640590 0x55f9d5a50bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:29.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.294+0000 7f57889cb740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d5661270 0x55f9d5a2a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:29.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.294+0000 7f57889cb740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f577c1f8ea0 con 0x55f9d5661270 2024-09-17T03:14:29.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.294+0000 7f57889cb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f577c15b6b0 con 0x55f9d56383b0 2024-09-17T03:14:29.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.294+0000 7f57889cb740 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f577c15bd00 con 0x55f9d5640590 2024-09-17T03:14:29.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.294+0000 7f577b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d5661270 0x55f9d5a2a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:29.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.295+0000 7f577b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d5661270 0x55f9d5a2a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50990/0 (socket says 172.21.15.87:50990) 2024-09-17T03:14:29.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.295+0000 7f577b7fe640 1 -- 172.21.15.87:0/1337335357 learned_addr learned my addr 172.21.15.87:0/1337335357 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:29.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.295+0000 7f577affd640 1 --2- 172.21.15.87:0/1337335357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9d56383b0 0x55f9d5a08460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:29.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.295+0000 7f57727fc640 1 --2- 172.21.15.87:0/1337335357 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55f9d5640590 0x55f9d5a50bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:29.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.295+0000 7f577b7fe640 1 -- 172.21.15.87:0/1337335357 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55f9d5640590 msgr2=0x55f9d5a50bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:14:29.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.295+0000 7f577b7fe640 1 --2- 172.21.15.87:0/1337335357 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55f9d5640590 0x55f9d5a50bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.295+0000 7f577b7fe640 1 -- 172.21.15.87:0/1337335357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9d56383b0 msgr2=0x55f9d5a08460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:14:29.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.295+0000 7f577b7fe640 1 --2- 172.21.15.87:0/1337335357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9d56383b0 0x55f9d5a08460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.295+0000 7f577b7fe640 1 -- 172.21.15.87:0/1337335357 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9d5a06710 con 0x55f9d5661270 2024-09-17T03:14:29.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.295+0000 7f577affd640 1 --2- 172.21.15.87:0/1337335357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9d56383b0 0x55f9d5a08460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:14:29.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.295+0000 7f577b7fe640 1 --2- 172.21.15.87:0/1337335357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d5661270 0x55f9d5a2a8f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5774009b30 tx=0x7f57740bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b8813787756982 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:29.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.295+0000 7f5770ff9640 1 -- 172.21.15.87:0/1337335357 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774084180 con 0x55f9d5661270 2024-09-17T03:14:29.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.295+0000 7f5770ff9640 1 -- 172.21.15.87:0/1337335357 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f577406f070 con 0x55f9d5661270 2024-09-17T03:14:29.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.295+0000 7f5770ff9640 1 -- 172.21.15.87:0/1337335357 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774088bf0 con 0x55f9d5661270 2024-09-17T03:14:29.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.296+0000 7f57889cb740 1 -- 172.21.15.87:0/1337335357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d5661270 msgr2=0x55f9d5a2a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:29.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.296+0000 7f57889cb740 1 --2- 172.21.15.87:0/1337335357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d5661270 0x55f9d5a2a8f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5774009b30 tx=0x7f57740bc040 comp rx=0 tx=0).stop 2024-09-17T03:14:29.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.296+0000 7f57889cb740 1 -- 172.21.15.87:0/1337335357 shutdown_connections 2024-09-17T03:14:29.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.296+0000 7f57889cb740 1 --2- 172.21.15.87:0/1337335357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d5661270 0x55f9d5a2a8f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.296+0000 7f57889cb740 1 --2- 172.21.15.87:0/1337335357 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55f9d5640590 0x55f9d5a50bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.296+0000 7f57889cb740 1 --2- 172.21.15.87:0/1337335357 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9d56383b0 0x55f9d5a08460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.296+0000 7f57889cb740 1 -- 172.21.15.87:0/1337335357 >> 172.21.15.87:0/1337335357 conn(0x55f9d56553c0 msgr2=0x55f9d5a03dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:29.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.296+0000 7f57889cb740 1 -- 172.21.15.87:0/1337335357 shutdown_connections 2024-09-17T03:14:29.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.296+0000 7f57889cb740 1 -- 172.21.15.87:0/1337335357 wait complete. 2024-09-17T03:14:29.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.297+0000 7f57889cb740 1 -- start start 2024-09-17T03:14:29.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.297+0000 7f57889cb740 1 Processor -- start 2024-09-17T03:14:29.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.298+0000 7f57889cb740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d56383b0 0x55f9d5a712c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:29.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.298+0000 7f57889cb740 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55f9d5640590 0x55f9d5a46210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:29.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.298+0000 7f57889cb740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9d5661270 0x55f9d5ab3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:29.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.298+0000 7f57889cb740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55f9d5a2bab0 con 0x55f9d56383b0 2024-09-17T03:14:29.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.298+0000 7f57889cb740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x55f9d5a2b610 con 0x55f9d5661270 2024-09-17T03:14:29.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.298+0000 7f57889cb740 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x55f9d5a47550 con 0x55f9d5640590 2024-09-17T03:14:29.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.298+0000 7f57889cb740 10 client.0 _ll_register_callbacks cb 0x7f576c0013e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:14:29.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.298+0000 7f577affd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d56383b0 0x55f9d5a712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:29.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.298+0000 7f577affd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d56383b0 0x55f9d5a712c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51006/0 (socket says 172.21.15.87:51006) 2024-09-17T03:14:29.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.298+0000 7f577affd640 1 -- 172.21.15.87:0/2383148182 learned_addr learned my addr 172.21.15.87:0/2383148182 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:29.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.299+0000 7f577affd640 1 -- 172.21.15.87:0/2383148182 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55f9d5640590 msgr2=0x55f9d5a46210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:14:29.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.299+0000 7f57727fc640 1 --2- 172.21.15.87:0/2383148182 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55f9d5640590 0x55f9d5a46210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:29.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.299+0000 7f577b7fe640 1 --2- 172.21.15.87:0/2383148182 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9d5661270 0x55f9d5ab3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:29.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.299+0000 7f577affd640 1 --2- 172.21.15.87:0/2383148182 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55f9d5640590 0x55f9d5a46210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.299+0000 7f577affd640 1 -- 172.21.15.87:0/2383148182 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9d5661270 msgr2=0x55f9d5ab3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:14:29.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.299+0000 7f577affd640 1 --2- 172.21.15.87:0/2383148182 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9d5661270 0x55f9d5ab3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.299+0000 7f577affd640 1 -- 172.21.15.87:0/2383148182 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9d5a2a020 con 0x55f9d56383b0 2024-09-17T03:14:29.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.299+0000 7f577affd640 1 --2- 172.21.15.87:0/2383148182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d56383b0 0x55f9d5a712c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f576c00e6c0 tx=0x7f576c0216d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b82c3ecfe0f9970 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:29.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.299+0000 7f575a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-09-17T03:14:29.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.299+0000 7f575a7fc640 1 -- 172.21.15.87:0/2383148182 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f576c036020 con 0x55f9d56383b0 2024-09-17T03:14:29.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.299+0000 7f575a7fc640 1 -- 172.21.15.87:0/2383148182 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f576c010040 con 0x55f9d56383b0 2024-09-17T03:14:29.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.299+0000 7f57889cb740 1 -- 172.21.15.87:0/2383148182 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f577406f070 con 0x55f9d56383b0 2024-09-17T03:14:29.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.299+0000 7f575a7fc640 1 -- 172.21.15.87:0/2383148182 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f576c040050 con 0x55f9d56383b0 2024-09-17T03:14:29.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.299+0000 7f57889cb740 10 client.4758 fetch_fsmap learned FSMap version 6 2024-09-17T03:14:29.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.299+0000 7f57889cb740 1 -- 172.21.15.87:0/2383148182 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55f9d5ab3b40 con 0x55f9d56383b0 2024-09-17T03:14:29.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.299+0000 7f575a7fc640 1 -- 172.21.15.87:0/2383148182 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f577406f070 con 0x55f9d56383b0 2024-09-17T03:14:29.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.299+0000 7f575a7fc640 1 -- 172.21.15.87:0/2383148182 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 6) ==== 3502+0+0 (secure 0 0 0) 0x7f576c043020 con 0x55f9d56383b0 2024-09-17T03:14:29.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.299+0000 7f57889cb740 10 client.4758 fetch_fsmap finished waiting for FSMap version 6 2024-09-17T03:14:29.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.299+0000 7f57889cb740 10 client.4758 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-17T03:14:29.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.299+0000 7f57889cb740 20 client.4758 populate_metadata read hostname 'smithi087' 2024-09-17T03:14:29.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.299+0000 7f57889cb740 1 --2- 172.21.15.87:0/2383148182 >> [v2:172.21.15.87:6834/1831073338,v1:172.21.15.87:6835/1831073338] conn(0x55f9d5a22930 0x55f9d5ad17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:29.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.299+0000 7f57889cb740 4 client.4758 mds_command: new command op to 4398 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T03:14:29.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.299+0000 7f57889cb740 1 -- 172.21.15.87:0/2383148182 --> [v2:172.21.15.87:6834/1831073338,v1:172.21.15.87:6835/1831073338] -- command(tid 0: {"prefix": "status"}) -- 0x55f9d5979220 con 0x55f9d5a22930 2024-09-17T03:14:29.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.300+0000 7f57727fc640 1 --2- 172.21.15.87:0/2383148182 >> [v2:172.21.15.87:6834/1831073338,v1:172.21.15.87:6835/1831073338] conn(0x55f9d5a22930 0x55f9d5ad17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:29.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.300+0000 7f575a7fc640 10 client.4758 ms_handle_connect on v2:172.21.15.87:6834/1831073338 2024-09-17T03:14:29.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.300+0000 7f57727fc640 1 --2- 172.21.15.87:0/2383148182 >> [v2:172.21.15.87:6834/1831073338,v1:172.21.15.87:6835/1831073338] conn(0x55f9d5a22930 0x55f9d5ad17b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b1d45c1f12a2632 server_cookie=c997750446ac3e6d in_seq=0 out_seq=0 2024-09-17T03:14:29.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.300+0000 7f575a7fc640 1 -- 172.21.15.87:0/2383148182 <== mds.0 v2:172.21.15.87:6834/1831073338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55f9d5979220 con 0x55f9d5a22930 2024-09-17T03:14:29.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.300+0000 7f575a7fc640 10 client.4758 handle_command_reply: tid=0 2024-09-17T03:14:29.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.300+0000 7f57889cb740 1 client.4758 shutdown 2024-09-17T03:14:29.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.300+0000 7f57889cb740 1 -- 172.21.15.87:0/2383148182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d56383b0 msgr2=0x55f9d5a712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:29.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.300+0000 7f57889cb740 1 --2- 172.21.15.87:0/2383148182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d56383b0 0x55f9d5a712c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f576c00e6c0 tx=0x7f576c0216d0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.300+0000 7f57889cb740 1 -- 172.21.15.87:0/2383148182 shutdown_connections 2024-09-17T03:14:29.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.300+0000 7f57889cb740 1 --2- 172.21.15.87:0/2383148182 >> [v2:172.21.15.87:6834/1831073338,v1:172.21.15.87:6835/1831073338] conn(0x55f9d5a22930 0x55f9d5ad17b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.300+0000 7f57889cb740 1 --2- 172.21.15.87:0/2383148182 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x55f9d5661270 0x55f9d5ab3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.300+0000 7f57889cb740 1 --2- 172.21.15.87:0/2383148182 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x55f9d5640590 0x55f9d5a46210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.300+0000 7f575a7fc640 0 client.4758 ms_handle_reset on v2:172.21.15.87:6834/1831073338 2024-09-17T03:14:29.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.300+0000 7f57889cb740 1 --2- 172.21.15.87:0/2383148182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f9d56383b0 0x55f9d5a712c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.300+0000 7f57889cb740 1 -- 172.21.15.87:0/2383148182 >> 172.21.15.87:0/2383148182 conn(0x55f9d56553c0 msgr2=0x55f9d5a03e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:29.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.300+0000 7f57889cb740 1 -- 172.21.15.87:0/2383148182 shutdown_connections 2024-09-17T03:14:29.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.300+0000 7f57889cb740 1 -- 172.21.15.87:0/2383148182 wait complete. 2024-09-17T03:14:29.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.300+0000 7f57889cb740 20 client.4758 trim_cache size 0 max 16384 2024-09-17T03:14:29.306 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-17T03:14:29.306 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "7b2553aa-e562-4da8-b175-8e91f460da55", 2024-09-17T03:14:29.306 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-09-17T03:14:29.307 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 4398, 2024-09-17T03:14:29.307 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-09-17T03:14:29.307 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-09-17T03:14:29.307 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-09-17T03:14:29.307 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 14.985882066, 2024-09-17T03:14:29.307 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 6, 2024-09-17T03:14:29.307 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 23, 2024-09-17T03:14:29.307 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 23, 2024-09-17T03:14:29.307 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 21.023121075999999 2024-09-17T03:14:29.307 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-17T03:14:29.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.301+0000 7f5781982640 1 -- 172.21.15.87:0/1114566348 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f575c078bc0 msgr2=0x7f575c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:29.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.301+0000 7f5781982640 1 --2- 172.21.15.87:0/1114566348 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f575c078bc0 0x7f575c07b0d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f576c0046b0 tx=0x7f576c009040 comp rx=0 tx=0).stop 2024-09-17T03:14:29.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.301+0000 7f5781982640 1 -- 172.21.15.87:0/1114566348 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f577c0d4860 msgr2=0x7f577c15e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:29.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.301+0000 7f5781982640 1 --2- 172.21.15.87:0/1114566348 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f577c0d4860 0x7f577c15e8c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5774006010 tx=0x7f5774003ba0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.301+0000 7f5781982640 1 -- 172.21.15.87:0/1114566348 shutdown_connections 2024-09-17T03:14:29.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.301+0000 7f5781982640 1 --2- 172.21.15.87:0/1114566348 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f575c078bc0 0x7f575c07b0d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.301+0000 7f5781982640 1 --2- 172.21.15.87:0/1114566348 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f577c0d4860 0x7f577c15e8c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.301+0000 7f5781982640 1 --2- 172.21.15.87:0/1114566348 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f577c0d3060 0x7f577c0d1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.301+0000 7f5781982640 1 --2- 172.21.15.87:0/1114566348 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f577c0d2c50 0x7f577c09e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.301+0000 7f5781982640 1 -- 172.21.15.87:0/1114566348 >> 172.21.15.87:0/1114566348 conn(0x7f577c08dae0 msgr2=0x7f577c161300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:29.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.301+0000 7f5781982640 1 -- 172.21.15.87:0/1114566348 shutdown_connections 2024-09-17T03:14:29.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.301+0000 7f5781982640 1 -- 172.21.15.87:0/1114566348 wait complete. 2024-09-17T03:14:29.311 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-17T03:14:29.338 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-17T03:14:29.338 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-17T03:14:29.338 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-17T03:14:29.339 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T03:14:29.339 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T03:14:29.339 INFO:tasks.ceph_fuse:client.2 config is {} 2024-09-17T03:14:29.339 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T03:14:29.339 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-17T03:14:29.339 INFO:tasks.ceph_fuse:client.1 config is {} 2024-09-17T03:14:29.339 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T03:14:29.339 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T03:14:29.339 INFO:tasks.ceph_fuse:client.3 config is {} 2024-09-17T03:14:29.339 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-17T03:14:29.339 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-17T03:14:29.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:29.340 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-09-17T03:14:29.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:29.356 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link delete ceph-brx 2024-09-17T03:14:29.422 INFO:teuthology.orchestra.run.smithi087.stderr:Cannot find device "ceph-brx" 2024-09-17T03:14:29.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:14:29.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:29.424 DEBUG:teuthology.orchestra.run.smithi149:> ip netns list 2024-09-17T03:14:29.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:29.443 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link delete ceph-brx 2024-09-17T03:14:29.516 INFO:teuthology.orchestra.run.smithi149.stderr:Cannot find device "ceph-brx" 2024-09-17T03:14:29.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:14:29.517 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-17T03:14:29.517 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-17T03:14:29.518 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T03:14:29.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.630+0000 7f1abf094640 1 -- 172.21.15.87:0/1103744389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ab80d1470 msgr2=0x7f1ab81673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:29.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.630+0000 7f1abf094640 1 --2- 172.21.15.87:0/1103744389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ab80d1470 0x7f1ab81673c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0005fd0 tx=0x7f1ab002a8b0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.630+0000 7f1abf094640 1 -- 172.21.15.87:0/1103744389 shutdown_connections 2024-09-17T03:14:29.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.630+0000 7f1abf094640 1 --2- 172.21.15.87:0/1103744389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ab80d1470 0x7f1ab81673c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.630+0000 7f1abf094640 1 -- 172.21.15.87:0/1103744389 >> 172.21.15.87:0/1103744389 conn(0x7f1ab8069360 msgr2=0x7f1ab80ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:29.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.630+0000 7f1abf094640 1 -- 172.21.15.87:0/1103744389 shutdown_connections 2024-09-17T03:14:29.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.630+0000 7f1abf094640 1 -- 172.21.15.87:0/1103744389 wait complete. 2024-09-17T03:14:29.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.632+0000 7f1abf094640 1 Processor -- start 2024-09-17T03:14:29.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.632+0000 7f1abf094640 1 -- start start 2024-09-17T03:14:29.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.633+0000 7f1abf094640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ab80b3d00 0x7f1ab81f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:29.633 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.633+0000 7f1abf094640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ab80d1470 0x7f1ab8159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:29.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.633+0000 7f1abf094640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ab815c6c0 0x7f1ab8159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:29.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.633+0000 7f1abf094640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab8167a80 con 0x7f1ab80d1470 2024-09-17T03:14:29.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.633+0000 7f1abf094640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab8167900 con 0x7f1ab80b3d00 2024-09-17T03:14:29.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.634+0000 7f1abf094640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab8167c00 con 0x7f1ab815c6c0 2024-09-17T03:14:29.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.634+0000 7f1aaffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ab80d1470 0x7f1ab8159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:29.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.634+0000 7f1aaffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ab80d1470 0x7f1ab8159a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51036/0 (socket says 172.21.15.87:51036) 2024-09-17T03:14:29.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.634+0000 7f1aaffff640 1 -- 172.21.15.87:0/2394985151 learned_addr learned my addr 172.21.15.87:0/2394985151 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:29.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.634+0000 7f1abce30640 1 --2- 172.21.15.87:0/2394985151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ab80b3d00 0x7f1ab81f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:29.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.634+0000 7f1abd631640 1 --2- 172.21.15.87:0/2394985151 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ab815c6c0 0x7f1ab8159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:29.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.634+0000 7f1aaffff640 1 -- 172.21.15.87:0/2394985151 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ab815c6c0 msgr2=0x7f1ab8159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:29.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.634+0000 7f1aaffff640 1 --2- 172.21.15.87:0/2394985151 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ab815c6c0 0x7f1ab8159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.634+0000 7f1aaffff640 1 -- 172.21.15.87:0/2394985151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ab80b3d00 msgr2=0x7f1ab81f73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:29.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.634+0000 7f1aaffff640 1 --2- 172.21.15.87:0/2394985151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ab80b3d00 0x7f1ab81f73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.634+0000 7f1aaffff640 1 -- 172.21.15.87:0/2394985151 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab8141c60 con 0x7f1ab80d1470 2024-09-17T03:14:29.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.634+0000 7f1aaffff640 1 --2- 172.21.15.87:0/2394985151 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ab80d1470 0x7f1ab8159a20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800c370 tx=0x7f1aa800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:29.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.634+0000 7f1aad7fa640 1 -- 172.21.15.87:0/2394985151 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa80052d0 con 0x7f1ab80d1470 2024-09-17T03:14:29.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.634+0000 7f1aad7fa640 1 -- 172.21.15.87:0/2394985151 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1aa8007020 con 0x7f1ab80d1470 2024-09-17T03:14:29.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.634+0000 7f1abf094640 1 -- 172.21.15.87:0/2394985151 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab8141f40 con 0x7f1ab80d1470 2024-09-17T03:14:29.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.634+0000 7f1aad7fa640 1 -- 172.21.15.87:0/2394985151 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8007470 con 0x7f1ab80d1470 2024-09-17T03:14:29.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.634+0000 7f1abf094640 1 -- 172.21.15.87:0/2394985151 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab8142500 con 0x7f1ab80d1470 2024-09-17T03:14:29.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.635+0000 7f1abf094640 1 -- 172.21.15.87:0/2394985151 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab8160c20 con 0x7f1ab80d1470 2024-09-17T03:14:29.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.636+0000 7f1aad7fa640 1 -- 172.21.15.87:0/2394985151 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1aa80067d0 con 0x7f1ab80d1470 2024-09-17T03:14:29.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.636+0000 7f1aad7fa640 1 --2- 172.21.15.87:0/2394985151 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1aa4078b90 0x7f1aa407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:29.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.637+0000 7f1aad7fa640 1 -- 172.21.15.87:0/2394985151 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1aa809a0e0 con 0x7f1ab80d1470 2024-09-17T03:14:29.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.637+0000 7f1abce30640 1 --2- 172.21.15.87:0/2394985151 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1aa4078b90 0x7f1aa407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:29.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.637+0000 7f1abce30640 1 --2- 172.21.15.87:0/2394985151 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1aa4078b90 0x7f1aa407b0a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40046b0 tx=0x7f1ab4007e90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:29.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.641+0000 7f1aad7fa640 1 -- 172.21.15.87:0/2394985151 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1aa8061f60 con 0x7f1ab80d1470 2024-09-17T03:14:29.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.851+0000 7f1abf094640 1 -- 172.21.15.87:0/2394985151 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1ab80cecc0 con 0x7f1ab80d1470 2024-09-17T03:14:29.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.852+0000 7f1aad7fa640 1 -- 172.21.15.87:0/2394985151 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7f1aa8066990 con 0x7f1ab80d1470 2024-09-17T03:14:29.851 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T03:14:29.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.854+0000 7f1abf094640 1 -- 172.21.15.87:0/2394985151 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1aa4078b90 msgr2=0x7f1aa407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:29.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.854+0000 7f1abf094640 1 --2- 172.21.15.87:0/2394985151 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1aa4078b90 0x7f1aa407b0a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40046b0 tx=0x7f1ab4007e90 comp rx=0 tx=0).stop 2024-09-17T03:14:29.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.854+0000 7f1abf094640 1 -- 172.21.15.87:0/2394985151 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ab80d1470 msgr2=0x7f1ab8159a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:29.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.854+0000 7f1abf094640 1 --2- 172.21.15.87:0/2394985151 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ab80d1470 0x7f1ab8159a20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800c370 tx=0x7f1aa800c830 comp rx=0 tx=0).stop 2024-09-17T03:14:29.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.854+0000 7f1abf094640 1 -- 172.21.15.87:0/2394985151 shutdown_connections 2024-09-17T03:14:29.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.854+0000 7f1abf094640 1 --2- 172.21.15.87:0/2394985151 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1aa4078b90 0x7f1aa407b0a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.854+0000 7f1abf094640 1 --2- 172.21.15.87:0/2394985151 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f1ab815c6c0 0x7f1ab8159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.854+0000 7f1abf094640 1 --2- 172.21.15.87:0/2394985151 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ab80d1470 0x7f1ab8159a20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.854+0000 7f1abf094640 1 --2- 172.21.15.87:0/2394985151 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1ab80b3d00 0x7f1ab81f73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:29.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.854+0000 7f1abf094640 1 -- 172.21.15.87:0/2394985151 >> 172.21.15.87:0/2394985151 conn(0x7f1ab8069360 msgr2=0x7f1ab8075d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:29.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.854+0000 7f1abf094640 1 -- 172.21.15.87:0/2394985151 shutdown_connections 2024-09-17T03:14:29.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29.854+0000 7f1abf094640 1 -- 172.21.15.87:0/2394985151 wait complete. 2024-09-17T03:14:29.863 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T03:14:29.863 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T03:14:29.864 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-09-17T03:14:29.864 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T03:14:29.864 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T03:14:29.864 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T03:14:29.864 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T03:14:29.864 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T03:14:29.864 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:14:29.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:29.864 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-09-17T03:14:29.880 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:14:29.880 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:14:29.880 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:14:29.880 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:29.880 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-09-17T03:14:29.880 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:29.880 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:14:29.880 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-09-17T03:14:29.880 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-09-17T03:14:29.881 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:14:29.881 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-09-17T03:14:29.881 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-09-17T03:14:29.881 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:14:29.881 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:14:29.881 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T03:14:29.881 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:14:29.881 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T03:14:29.881 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42762sec preferred_lft 42762sec 2024-09-17T03:14:29.881 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link noprefixroute 2024-09-17T03:14:29.881 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:29.882 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-17T03:14:29.882 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:14:29.882 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-09-17T03:14:29.882 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link add name ceph-brx type bridge 2024-09-17T03:14:29.882 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip addr flush dev ceph-brx 2024-09-17T03:14:29.882 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set ceph-brx up 2024-09-17T03:14:29.882 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-17T03:14:29.882 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-09-17T03:14:29.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:14:30.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:14:30.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:30.108 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T03:14:30.135 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-09-17T03:14:30.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:30.136 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-09-17T03:14:30.201 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-09-17T03:14:30.201 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T03:14:30.201 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T03:14:30.202 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T03:14:30.202 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T03:14:30.202 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:14:30.202 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-09-17T03:14:30.202 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T03:14:30.202 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T03:14:30.202 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T03:14:30.203 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-09-17T03:14:30.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:14:30.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:14:30.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:30.496 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-09-17T03:14:30.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:30.511 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list-id 2024-09-17T03:14:30.568 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:14:30.568 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-09-17T03:14:30.568 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:14:30.568 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-17T03:14:30.568 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-09-17T03:14:30.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:14:30.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:14:30.688 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-17T03:14:30.689 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:14:30.689 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-09-17T03:14:30.689 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-17T03:14:30.689 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-17T03:14:30.689 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-17T03:14:30.689 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-17T03:14:30.689 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-17T03:14:30.689 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-09-17T03:14:30.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:14:30.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:14:30.821 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:14:30.821 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-09-17T03:14:30.821 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-09-17T03:14:30.821 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set dev brx.0 master ceph-brx 2024-09-17T03:14:30.821 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-09-17T03:14:30.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:14:30.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:14:30.908 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T03:14:30.909 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:14:30.909 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T03:14:30.923 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:14:30.924 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:14:30.924 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:14:30.978 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-09-17T03:14:31.044 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T03:14:31.099 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-09-17T03:14:31.100 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-09-17T03:14:31.100 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-09-17T03:14:31.100 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-09-17T03:14:31.100 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-09-17T03:14:31.100 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-09-17T03:14:31.100 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-09-17T03:14:31.100 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-09-17T03:14:31.100 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-09-17T03:14:31.100 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-09-17T03:14:31.100 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-09-17T03:14:31.100 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-09-17T03:14:31.100 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T03:14:31.101 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-09-17T03:14:31.101 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-09-17T03:14:31.101 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-09-17T03:14:31.101 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-09-17T03:14:31.101 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-09-17T03:14:31.101 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-09-17T03:14:31.101 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T03:14:31.101 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T03:14:31.101 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T03:14:31.101 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T03:14:31.101 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-09-17T03:14:31.101 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-09-17T03:14:31.101 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T03:14:31.101 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T03:14:31.102 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T03:14:31.102 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T03:14:31.102 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-09-17T03:14:31.102 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:14:31.102 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:14:31.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:31.102 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-09-17T03:14:31.154 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T03:14:31.155 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-17T03:14:31.197 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-09-17T03:14:31.221 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T03:14:31.279 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-09-17T03:14:31.279 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-09-17T03:14:31.279 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-09-17T03:14:31.280 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-09-17T03:14:31.280 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-09-17T03:14:31.280 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-09-17T03:14:31.280 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-09-17T03:14:31.280 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-09-17T03:14:31.280 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-09-17T03:14:31.280 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-09-17T03:14:31.280 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-09-17T03:14:31.280 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-09-17T03:14:31.280 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T03:14:31.280 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-09-17T03:14:31.280 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-09-17T03:14:31.281 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-09-17T03:14:31.281 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-09-17T03:14:31.281 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-09-17T03:14:31.281 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-09-17T03:14:31.281 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T03:14:31.281 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T03:14:31.281 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T03:14:31.281 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T03:14:31.281 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-09-17T03:14:31.281 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-09-17T03:14:31.281 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T03:14:31.281 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T03:14:31.281 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T03:14:31.281 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T03:14:31.282 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-09-17T03:14:31.282 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:14:31.282 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:14:31.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:31.282 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-09-17T03:14:31.307 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:2024-09-17T03:14:31.308+0000 7ff450701480 -1 init, newargv = 0x7ff434001080 newargc=15 2024-09-17T03:14:31.308 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:ceph-fuse[45959]: starting ceph client 2024-09-17T03:14:31.313 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:ceph-fuse[45959]: starting fuse 2024-09-17T03:14:31.323 INFO:teuthology.orchestra.run.smithi087.stdout:42 2024-09-17T03:14:31.323 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-17T03:14:31.323 DEBUG:teuthology.orchestra.run.smithi087:> sudo stdin-killer -- python3 -c ' 2024-09-17T03:14:31.323 DEBUG:teuthology.orchestra.run.smithi087:> import glob 2024-09-17T03:14:31.323 DEBUG:teuthology.orchestra.run.smithi087:> import re 2024-09-17T03:14:31.324 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-09-17T03:14:31.324 DEBUG:teuthology.orchestra.run.smithi087:> import subprocess 2024-09-17T03:14:31.324 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:14:31.324 DEBUG:teuthology.orchestra.run.smithi087:> def _find_admin_socket(client_name): 2024-09-17T03:14:31.324 DEBUG:teuthology.orchestra.run.smithi087:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T03:14:31.324 DEBUG:teuthology.orchestra.run.smithi087:> files = glob.glob(asok_path) 2024-09-17T03:14:31.324 DEBUG:teuthology.orchestra.run.smithi087:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T03:14:31.324 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:14:31.324 DEBUG:teuthology.orchestra.run.smithi087:> # Given a non-glob path, it better be there 2024-09-17T03:14:31.324 DEBUG:teuthology.orchestra.run.smithi087:> if "*" not in asok_path: 2024-09-17T03:14:31.324 DEBUG:teuthology.orchestra.run.smithi087:> assert(len(files) == 1) 2024-09-17T03:14:31.324 DEBUG:teuthology.orchestra.run.smithi087:> return files[0] 2024-09-17T03:14:31.324 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:14:31.324 DEBUG:teuthology.orchestra.run.smithi087:> for f in files: 2024-09-17T03:14:31.325 DEBUG:teuthology.orchestra.run.smithi087:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T03:14:31.325 DEBUG:teuthology.orchestra.run.smithi087:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T03:14:31.325 DEBUG:teuthology.orchestra.run.smithi087:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T03:14:31.325 DEBUG:teuthology.orchestra.run.smithi087:> contents = proc_f.read() 2024-09-17T03:14:31.325 DEBUG:teuthology.orchestra.run.smithi087:> if mountpoint in contents: 2024-09-17T03:14:31.325 DEBUG:teuthology.orchestra.run.smithi087:> return f 2024-09-17T03:14:31.325 DEBUG:teuthology.orchestra.run.smithi087:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T03:14:31.325 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:14:31.325 DEBUG:teuthology.orchestra.run.smithi087:> print(_find_admin_socket("client.0")) 2024-09-17T03:14:31.325 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-09-17T03:14:31.441 INFO:teuthology.orchestra.run.smithi087.stdout:/var/run/ceph/ceph-client.0.45959.asok 2024-09-17T03:14:31.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:14:31.451 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.45959.asok 2024-09-17T03:14:31.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:31.451 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.45959.asok status 2024-09-17T03:14:31.593 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-17T03:14:31.593 INFO:teuthology.orchestra.run.smithi087.stdout: "metadata": { 2024-09-17T03:14:31.593 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T03:14:31.593 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T03:14:31.593 INFO:teuthology.orchestra.run.smithi087.stdout: "entity_id": "0", 2024-09-17T03:14:31.593 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": "smithi087", 2024-09-17T03:14:31.594 INFO:teuthology.orchestra.run.smithi087.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T03:14:31.594 INFO:teuthology.orchestra.run.smithi087.stdout: "pid": "45959", 2024-09-17T03:14:31.594 INFO:teuthology.orchestra.run.smithi087.stdout: "root": "/" 2024-09-17T03:14:31.594 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:31.594 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_count": 0, 2024-09-17T03:14:31.594 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_pinned_count": 0, 2024-09-17T03:14:31.594 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 4776, 2024-09-17T03:14:31.594 INFO:teuthology.orchestra.run.smithi087.stdout: "inst": { 2024-09-17T03:14:31.594 INFO:teuthology.orchestra.run.smithi087.stdout: "name": { 2024-09-17T03:14:31.594 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "client", 2024-09-17T03:14:31.594 INFO:teuthology.orchestra.run.smithi087.stdout: "num": 4776 2024-09-17T03:14:31.594 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:31.595 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-09-17T03:14:31.595 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-09-17T03:14:31.595 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-09-17T03:14:31.595 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 1324266883 2024-09-17T03:14:31.595 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:31.595 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:31.595 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-09-17T03:14:31.595 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-09-17T03:14:31.595 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-09-17T03:14:31.595 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 1324266883 2024-09-17T03:14:31.595 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:31.595 INFO:teuthology.orchestra.run.smithi087.stdout: "inst_str": "client.4776 192.168.0.1:0/1324266883", 2024-09-17T03:14:31.595 INFO:teuthology.orchestra.run.smithi087.stdout: "addr_str": "192.168.0.1:0/1324266883", 2024-09-17T03:14:31.595 INFO:teuthology.orchestra.run.smithi087.stdout: "inode_count": 1, 2024-09-17T03:14:31.596 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_epoch": 6, 2024-09-17T03:14:31.596 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch": 23, 2024-09-17T03:14:31.596 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch_barrier": 0, 2024-09-17T03:14:31.596 INFO:teuthology.orchestra.run.smithi087.stdout: "blocklisted": false, 2024-09-17T03:14:31.596 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs" 2024-09-17T03:14:31.596 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-17T03:14:31.601 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-17T03:14:31.601 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T03:14:31.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.715+0000 7f862500e640 1 -- 172.21.15.87:0/272347499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f860c004800 msgr2=0x7f860c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:31.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.715+0000 7f862500e640 1 --2- 172.21.15.87:0/272347499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f860c004800 0x7f860c004c10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8608009b50 tx=0x7f860c005830 comp rx=0 tx=0).stop 2024-09-17T03:14:31.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.715+0000 7f861f59e640 1 -- 172.21.15.87:0/272347499 reap_dead start 2024-09-17T03:14:31.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.715+0000 7f862500e640 1 -- 172.21.15.87:0/272347499 shutdown_connections 2024-09-17T03:14:31.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.715+0000 7f862500e640 1 -- 172.21.15.87:0/272347499 >> 172.21.15.87:0/272347499 conn(0x7f8620069360 msgr2=0x7f86200ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:31.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.715+0000 7f862500e640 1 -- 172.21.15.87:0/272347499 shutdown_connections 2024-09-17T03:14:31.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.715+0000 7f862500e640 1 -- 172.21.15.87:0/272347499 wait complete. 2024-09-17T03:14:31.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.717+0000 7f862500e640 1 Processor -- start 2024-09-17T03:14:31.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.717+0000 7f862500e640 1 -- start start 2024-09-17T03:14:31.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.718+0000 7f862500e640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f860c004800 0x7f86200cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:31.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.718+0000 7f862500e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86201584d0 0x7f862014b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:31.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.718+0000 7f862500e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f862014b980 0x7f862014a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:31.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.718+0000 7f862500e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f86200d0870 con 0x7f86201584d0 2024-09-17T03:14:31.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.718+0000 7f862500e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f86200d09f0 con 0x7f862014b980 2024-09-17T03:14:31.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.718+0000 7f862500e640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f86200d0b70 con 0x7f860c004800 2024-09-17T03:14:31.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.718+0000 7f8617fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86201584d0 0x7f862014b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:31.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.718+0000 7f8617fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86201584d0 0x7f862014b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51046/0 (socket says 172.21.15.87:51046) 2024-09-17T03:14:31.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.718+0000 7f8617fff640 1 -- 172.21.15.87:0/423643897 learned_addr learned my addr 172.21.15.87:0/423643897 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:31.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.719+0000 7f861ed9d640 1 --2- 172.21.15.87:0/423643897 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f860c004800 0x7f86200cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:31.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.719+0000 7f861f59e640 1 --2- 172.21.15.87:0/423643897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f862014b980 0x7f862014a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:31.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.719+0000 7f8617fff640 1 -- 172.21.15.87:0/423643897 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f860c004800 msgr2=0x7f86200cb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:31.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.719+0000 7f8617fff640 1 --2- 172.21.15.87:0/423643897 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f860c004800 0x7f86200cb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:31.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.719+0000 7f8617fff640 1 -- 172.21.15.87:0/423643897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f862014b980 msgr2=0x7f862014a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:31.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.719+0000 7f8617fff640 1 --2- 172.21.15.87:0/423643897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f862014b980 0x7f862014a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:31.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.719+0000 7f8617fff640 1 -- 172.21.15.87:0/423643897 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8620149620 con 0x7f86201584d0 2024-09-17T03:14:31.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.719+0000 7f8617fff640 1 --2- 172.21.15.87:0/423643897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86201584d0 0x7f862014b440 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8608009a70 tx=0x7f8608009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:31.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.719+0000 7f86177fe640 1 -- 172.21.15.87:0/423643897 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608038050 con 0x7f86201584d0 2024-09-17T03:14:31.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.719+0000 7f86177fe640 1 -- 172.21.15.87:0/423643897 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8608039040 con 0x7f86201584d0 2024-09-17T03:14:31.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.719+0000 7f862500e640 1 -- 172.21.15.87:0/423643897 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8620149900 con 0x7f86201584d0 2024-09-17T03:14:31.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.719+0000 7f86177fe640 1 -- 172.21.15.87:0/423643897 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8608033e40 con 0x7f86201584d0 2024-09-17T03:14:31.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.719+0000 7f862500e640 1 -- 172.21.15.87:0/423643897 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f862015d930 con 0x7f86201584d0 2024-09-17T03:14:31.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.720+0000 7f862500e640 1 -- 172.21.15.87:0/423643897 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86200b4110 con 0x7f86201584d0 2024-09-17T03:14:31.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.721+0000 7f86177fe640 1 -- 172.21.15.87:0/423643897 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f860802b8c0 con 0x7f86201584d0 2024-09-17T03:14:31.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.721+0000 7f86177fe640 1 --2- 172.21.15.87:0/423643897 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8600078b90 0x7f860007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:31.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.721+0000 7f86177fe640 1 -- 172.21.15.87:0/423643897 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f86080bbf40 con 0x7f86201584d0 2024-09-17T03:14:31.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.722+0000 7f861ed9d640 1 --2- 172.21.15.87:0/423643897 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8600078b90 0x7f860007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:31.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.722+0000 7f861ed9d640 1 --2- 172.21.15.87:0/423643897 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8600078b90 0x7f860007b0a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8610001a30 tx=0x7f86100011e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:31.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.726+0000 7f86177fe640 1 -- 172.21.15.87:0/423643897 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8608084410 con 0x7f86201584d0 2024-09-17T03:14:31.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.936+0000 7f862500e640 1 -- 172.21.15.87:0/423643897 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f862015ba90 con 0x7f86201584d0 2024-09-17T03:14:31.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.937+0000 7f86177fe640 1 -- 172.21.15.87:0/423643897 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7f8608083d10 con 0x7f86201584d0 2024-09-17T03:14:31.936 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T03:14:31.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.939+0000 7f862500e640 1 -- 172.21.15.87:0/423643897 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8600078b90 msgr2=0x7f860007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:31.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.939+0000 7f862500e640 1 --2- 172.21.15.87:0/423643897 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8600078b90 0x7f860007b0a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8610001a30 tx=0x7f86100011e0 comp rx=0 tx=0).stop 2024-09-17T03:14:31.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.939+0000 7f862500e640 1 -- 172.21.15.87:0/423643897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86201584d0 msgr2=0x7f862014b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:31.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.939+0000 7f862500e640 1 --2- 172.21.15.87:0/423643897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86201584d0 0x7f862014b440 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8608009a70 tx=0x7f8608009850 comp rx=0 tx=0).stop 2024-09-17T03:14:31.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.939+0000 7f862500e640 1 -- 172.21.15.87:0/423643897 shutdown_connections 2024-09-17T03:14:31.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.939+0000 7f862500e640 1 --2- 172.21.15.87:0/423643897 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8600078b90 0x7f860007b0a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:31.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.939+0000 7f862500e640 1 --2- 172.21.15.87:0/423643897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f862014b980 0x7f862014a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:31.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.939+0000 7f862500e640 1 --2- 172.21.15.87:0/423643897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86201584d0 0x7f862014b440 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:31.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.939+0000 7f862500e640 1 --2- 172.21.15.87:0/423643897 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f860c004800 0x7f86200cb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:31.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.939+0000 7f862500e640 1 -- 172.21.15.87:0/423643897 >> 172.21.15.87:0/423643897 conn(0x7f8620069360 msgr2=0x7f8620075c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:31.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.939+0000 7f862500e640 1 -- 172.21.15.87:0/423643897 shutdown_connections 2024-09-17T03:14:31.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:31.939+0000 7f862500e640 1 -- 172.21.15.87:0/423643897 wait complete. 2024-09-17T03:14:31.949 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T03:14:31.949 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T03:14:31.949 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-09-17T03:14:31.949 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-17T03:14:31.949 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-17T03:14:31.949 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T03:14:31.949 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T03:14:31.949 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T03:14:31.949 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-17T03:14:31.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:31.949 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-09-17T03:14:31.965 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:14:31.965 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:14:31.965 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:14:31.965 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:31.965 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-09-17T03:14:31.965 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:31.966 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:14:31.966 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-09-17T03:14:31.966 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-09-17T03:14:31.966 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:14:31.966 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-09-17T03:14:31.966 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-09-17T03:14:31.966 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:14:31.966 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:14:31.966 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T03:14:31.966 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:14:31.966 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T03:14:31.966 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42760sec preferred_lft 42760sec 2024-09-17T03:14:31.966 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link noprefixroute 2024-09-17T03:14:31.967 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:31.967 INFO:teuthology.orchestra.run.smithi087.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:14:31.967 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether b2:cb:c1:94:14:5d brd ff:ff:ff:ff:ff:ff 2024-09-17T03:14:31.967 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T03:14:31.967 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:31.967 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c822:ceff:fef0:4a99/64 scope link 2024-09-17T03:14:31.967 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:31.967 INFO:teuthology.orchestra.run.smithi087.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:14:31.967 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether b2:cb:c1:94:14:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:14:31.967 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::b0cb:c1ff:fe94:145d/64 scope link tentative 2024-09-17T03:14:31.967 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:31.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:31.968 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T03:14:32.029 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-09-17T03:14:32.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:32.031 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-09-17T03:14:32.087 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-09-17T03:14:32.087 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T03:14:32.087 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T03:14:32.087 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T03:14:32.088 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T03:14:32.088 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:14:32.088 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-09-17T03:14:32.088 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T03:14:32.088 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T03:14:32.088 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T03:14:32.088 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-09-17T03:14:32.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:14:32.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:14:32.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:32.213 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-09-17T03:14:32.229 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T03:14:32.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:32.229 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list-id 2024-09-17T03:14:32.285 INFO:teuthology.orchestra.run.smithi087.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-17T03:14:32.285 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:14:32.285 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-09-17T03:14:32.285 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:14:32.285 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-17T03:14:32.285 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-09-17T03:14:32.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:14:32.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:14:32.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:32.403 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-17T03:14:32.427 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:14:32.428 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:14:32.428 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:14:32.428 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:32.428 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-09-17T03:14:32.428 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:32.428 INFO:teuthology.orchestra.run.smithi087.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:14:32.428 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 66:0a:3e:4a:da:f6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T03:14:32.428 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T03:14:32.428 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:32.428 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::640a:3eff:fe4a:daf6/64 scope link tentative 2024-09-17T03:14:32.428 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:32.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:32.429 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-17T03:14:32.495 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:14:32.496 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:14:32.496 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:14:32.496 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:32.496 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-09-17T03:14:32.496 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:32.496 INFO:teuthology.orchestra.run.smithi087.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:14:32.497 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 66:0a:3e:4a:da:f6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T03:14:32.497 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T03:14:32.497 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:32.497 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::640a:3eff:fe4a:daf6/64 scope link tentative 2024-09-17T03:14:32.497 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:32.499 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-17T03:14:32.499 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:14:32.499 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-09-17T03:14:32.500 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-17T03:14:32.500 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-17T03:14:32.500 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-17T03:14:32.500 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-17T03:14:32.500 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-17T03:14:32.500 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-09-17T03:14:32.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:14:32.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:14:32.651 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:14:32.651 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-09-17T03:14:32.651 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.1 up 2024-09-17T03:14:32.651 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set dev brx.1 master ceph-brx 2024-09-17T03:14:32.651 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-09-17T03:14:32.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:14:32.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:14:32.727 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-09-17T03:14:32.727 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:14:32.727 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-17T03:14:32.755 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-17T03:14:32.755 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:14:32.755 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-17T03:14:32.810 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-09-17T03:14:32.876 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T03:14:32.932 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-09-17T03:14:32.933 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-09-17T03:14:32.933 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-09-17T03:14:32.933 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-09-17T03:14:32.933 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-09-17T03:14:32.933 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-09-17T03:14:32.933 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-09-17T03:14:32.933 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-09-17T03:14:32.933 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-09-17T03:14:32.933 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-09-17T03:14:32.933 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-09-17T03:14:32.933 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-09-17T03:14:32.933 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T03:14:32.934 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-09-17T03:14:32.934 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-09-17T03:14:32.934 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-09-17T03:14:32.934 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-09-17T03:14:32.934 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-09-17T03:14:32.934 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-09-17T03:14:32.934 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T03:14:32.934 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T03:14:32.934 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T03:14:32.934 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T03:14:32.934 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-09-17T03:14:32.934 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-09-17T03:14:32.934 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T03:14:32.935 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T03:14:32.935 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T03:14:32.935 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T03:14:32.935 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-09-17T03:14:32.935 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:14:32.935 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:14:32.935 INFO:teuthology.orchestra.run.smithi087.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-17T03:14:32.935 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:14:32.935 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:14:32.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:32.936 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-09-17T03:14:32.987 INFO:teuthology.orchestra.run.smithi087.stdout:42 2024-09-17T03:14:32.988 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-09-17T03:14:32.988 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-09-17T03:14:33.030 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-09-17T03:14:33.053 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T03:14:33.112 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-09-17T03:14:33.112 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-09-17T03:14:33.112 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-09-17T03:14:33.113 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-09-17T03:14:33.113 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-09-17T03:14:33.113 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-09-17T03:14:33.113 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-09-17T03:14:33.113 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-09-17T03:14:33.113 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-09-17T03:14:33.113 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-09-17T03:14:33.113 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-09-17T03:14:33.113 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-09-17T03:14:33.113 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T03:14:33.113 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-09-17T03:14:33.113 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-09-17T03:14:33.113 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-09-17T03:14:33.114 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-09-17T03:14:33.114 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-09-17T03:14:33.114 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-09-17T03:14:33.114 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T03:14:33.114 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T03:14:33.114 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T03:14:33.114 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T03:14:33.114 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-09-17T03:14:33.114 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-09-17T03:14:33.114 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T03:14:33.114 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T03:14:33.114 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T03:14:33.114 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T03:14:33.114 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-09-17T03:14:33.115 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:14:33.115 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:14:33.115 INFO:teuthology.orchestra.run.smithi087.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-17T03:14:33.115 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:14:33.115 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:14:33.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:33.115 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-09-17T03:14:33.121 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi087.stderr:2024-09-17T03:14:33.122+0000 7f918dc0e480 -1 init, newargv = 0x7f9170001f00 newargc=15 2024-09-17T03:14:33.121 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi087.stderr:ceph-fuse[46444]: starting ceph client 2024-09-17T03:14:33.125 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi087.stderr:ceph-fuse[46444]: starting fuse 2024-09-17T03:14:33.136 INFO:teuthology.orchestra.run.smithi087.stdout:42 2024-09-17T03:14:33.136 INFO:teuthology.orchestra.run.smithi087.stdout:44 2024-09-17T03:14:33.136 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 44] 2024-09-17T03:14:33.136 DEBUG:teuthology.orchestra.run.smithi087:> sudo stdin-killer -- python3 -c ' 2024-09-17T03:14:33.136 DEBUG:teuthology.orchestra.run.smithi087:> import glob 2024-09-17T03:14:33.136 DEBUG:teuthology.orchestra.run.smithi087:> import re 2024-09-17T03:14:33.136 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-09-17T03:14:33.137 DEBUG:teuthology.orchestra.run.smithi087:> import subprocess 2024-09-17T03:14:33.137 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:14:33.137 DEBUG:teuthology.orchestra.run.smithi087:> def _find_admin_socket(client_name): 2024-09-17T03:14:33.137 DEBUG:teuthology.orchestra.run.smithi087:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-09-17T03:14:33.137 DEBUG:teuthology.orchestra.run.smithi087:> files = glob.glob(asok_path) 2024-09-17T03:14:33.137 DEBUG:teuthology.orchestra.run.smithi087:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-09-17T03:14:33.137 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:14:33.137 DEBUG:teuthology.orchestra.run.smithi087:> # Given a non-glob path, it better be there 2024-09-17T03:14:33.137 DEBUG:teuthology.orchestra.run.smithi087:> if "*" not in asok_path: 2024-09-17T03:14:33.137 DEBUG:teuthology.orchestra.run.smithi087:> assert(len(files) == 1) 2024-09-17T03:14:33.137 DEBUG:teuthology.orchestra.run.smithi087:> return files[0] 2024-09-17T03:14:33.137 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:14:33.137 DEBUG:teuthology.orchestra.run.smithi087:> for f in files: 2024-09-17T03:14:33.137 DEBUG:teuthology.orchestra.run.smithi087:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T03:14:33.138 DEBUG:teuthology.orchestra.run.smithi087:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T03:14:33.138 DEBUG:teuthology.orchestra.run.smithi087:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T03:14:33.138 DEBUG:teuthology.orchestra.run.smithi087:> contents = proc_f.read() 2024-09-17T03:14:33.138 DEBUG:teuthology.orchestra.run.smithi087:> if mountpoint in contents: 2024-09-17T03:14:33.138 DEBUG:teuthology.orchestra.run.smithi087:> return f 2024-09-17T03:14:33.138 DEBUG:teuthology.orchestra.run.smithi087:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T03:14:33.138 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:14:33.138 DEBUG:teuthology.orchestra.run.smithi087:> print(_find_admin_socket("client.2")) 2024-09-17T03:14:33.138 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-09-17T03:14:33.253 INFO:teuthology.orchestra.run.smithi087.stdout:/var/run/ceph/ceph-client.2.46444.asok 2024-09-17T03:14:33.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:14:33.263 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.46444.asok 2024-09-17T03:14:33.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:33.264 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.46444.asok status 2024-09-17T03:14:33.405 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-17T03:14:33.405 INFO:teuthology.orchestra.run.smithi087.stdout: "metadata": { 2024-09-17T03:14:33.405 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T03:14:33.405 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T03:14:33.405 INFO:teuthology.orchestra.run.smithi087.stdout: "entity_id": "2", 2024-09-17T03:14:33.405 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": "smithi087", 2024-09-17T03:14:33.405 INFO:teuthology.orchestra.run.smithi087.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-09-17T03:14:33.406 INFO:teuthology.orchestra.run.smithi087.stdout: "pid": "46444", 2024-09-17T03:14:33.406 INFO:teuthology.orchestra.run.smithi087.stdout: "root": "/" 2024-09-17T03:14:33.406 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:33.406 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_count": 0, 2024-09-17T03:14:33.406 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_pinned_count": 0, 2024-09-17T03:14:33.406 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 4788, 2024-09-17T03:14:33.406 INFO:teuthology.orchestra.run.smithi087.stdout: "inst": { 2024-09-17T03:14:33.406 INFO:teuthology.orchestra.run.smithi087.stdout: "name": { 2024-09-17T03:14:33.406 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "client", 2024-09-17T03:14:33.406 INFO:teuthology.orchestra.run.smithi087.stdout: "num": 4788 2024-09-17T03:14:33.406 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:33.406 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-09-17T03:14:33.406 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-09-17T03:14:33.406 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.2:0", 2024-09-17T03:14:33.406 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 1627301987 2024-09-17T03:14:33.407 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:33.407 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:33.407 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-09-17T03:14:33.407 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-09-17T03:14:33.407 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.2:0", 2024-09-17T03:14:33.407 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 1627301987 2024-09-17T03:14:33.407 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:33.407 INFO:teuthology.orchestra.run.smithi087.stdout: "inst_str": "client.4788 192.168.0.2:0/1627301987", 2024-09-17T03:14:33.407 INFO:teuthology.orchestra.run.smithi087.stdout: "addr_str": "192.168.0.2:0/1627301987", 2024-09-17T03:14:33.407 INFO:teuthology.orchestra.run.smithi087.stdout: "inode_count": 1, 2024-09-17T03:14:33.407 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_epoch": 6, 2024-09-17T03:14:33.407 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch": 23, 2024-09-17T03:14:33.407 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch_barrier": 0, 2024-09-17T03:14:33.407 INFO:teuthology.orchestra.run.smithi087.stdout: "blocklisted": false, 2024-09-17T03:14:33.408 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs" 2024-09-17T03:14:33.408 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-17T03:14:33.413 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-17T03:14:33.413 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T03:14:33.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.527+0000 7f442eb4a640 1 -- 172.21.15.87:0/2353790137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44280b7f10 msgr2=0x7f442815c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:33.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.527+0000 7f442eb4a640 1 --2- 172.21.15.87:0/2353790137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44280b7f10 0x7f442815c5f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4418009830 tx=0x7f441802a480 comp rx=0 tx=0).stop 2024-09-17T03:14:33.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.527+0000 7f442eb4a640 1 -- 172.21.15.87:0/2353790137 shutdown_connections 2024-09-17T03:14:33.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.527+0000 7f442eb4a640 1 --2- 172.21.15.87:0/2353790137 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44280b3d60 0x7f44280b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:33.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.527+0000 7f442eb4a640 1 --2- 172.21.15.87:0/2353790137 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44280b7f10 0x7f442815c5f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:33.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.527+0000 7f442eb4a640 1 -- 172.21.15.87:0/2353790137 >> 172.21.15.87:0/2353790137 conn(0x7f4428069330 msgr2=0x7f44280ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:33.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.527+0000 7f442eb4a640 1 -- 172.21.15.87:0/2353790137 shutdown_connections 2024-09-17T03:14:33.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.527+0000 7f442eb4a640 1 -- 172.21.15.87:0/2353790137 wait complete. 2024-09-17T03:14:33.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.529+0000 7f442eb4a640 1 Processor -- start 2024-09-17T03:14:33.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.529+0000 7f442eb4a640 1 -- start start 2024-09-17T03:14:33.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.530+0000 7f442eb4a640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44280710e0 0x7f4428159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:33.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.530+0000 7f442eb4a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44280b3d60 0x7f44280cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:33.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.530+0000 7f442eb4a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44280b7f10 0x7f44280a9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:33.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.530+0000 7f442eb4a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f44280d1c70 con 0x7f44280b3d60 2024-09-17T03:14:33.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.530+0000 7f442eb4a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f44280b7b10 con 0x7f44280b7f10 2024-09-17T03:14:33.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.530+0000 7f442eb4a640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f44281f38d0 con 0x7f44280710e0 2024-09-17T03:14:33.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.531+0000 7f4427fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44280b3d60 0x7f44280cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:33.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.531+0000 7f4427fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44280b3d60 0x7f44280cb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51056/0 (socket says 172.21.15.87:51056) 2024-09-17T03:14:33.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.531+0000 7f4427fff640 1 -- 172.21.15.87:0/3027584611 learned_addr learned my addr 172.21.15.87:0/3027584611 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:33.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.531+0000 7f442d0e7640 1 --2- 172.21.15.87:0/3027584611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44280b7f10 0x7f44280a9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:33.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.531+0000 7f442c8e6640 1 --2- 172.21.15.87:0/3027584611 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44280710e0 0x7f4428159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:33.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.531+0000 7f4427fff640 1 -- 172.21.15.87:0/3027584611 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44280710e0 msgr2=0x7f4428159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:33.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.531+0000 7f4427fff640 1 --2- 172.21.15.87:0/3027584611 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44280710e0 0x7f4428159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:33.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.531+0000 7f4427fff640 1 -- 172.21.15.87:0/3027584611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44280b7f10 msgr2=0x7f44280a9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:33.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.531+0000 7f4427fff640 1 --2- 172.21.15.87:0/3027584611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44280b7f10 0x7f44280a9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:33.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.531+0000 7f4427fff640 1 -- 172.21.15.87:0/3027584611 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4428159f50 con 0x7f44280b3d60 2024-09-17T03:14:33.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.531+0000 7f442c8e6640 1 --2- 172.21.15.87:0/3027584611 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44280710e0 0x7f4428159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:33.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.531+0000 7f4427fff640 1 --2- 172.21.15.87:0/3027584611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44280b3d60 0x7f44280cb260 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f441c00ece0 tx=0x7f441c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:33.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.531+0000 7f44257fa640 1 -- 172.21.15.87:0/3027584611 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c003b70 con 0x7f44280b3d60 2024-09-17T03:14:33.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.531+0000 7f44257fa640 1 -- 172.21.15.87:0/3027584611 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f441c003d10 con 0x7f44280b3d60 2024-09-17T03:14:33.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.531+0000 7f44257fa640 1 -- 172.21.15.87:0/3027584611 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c004280 con 0x7f44280b3d60 2024-09-17T03:14:33.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.531+0000 7f442eb4a640 1 -- 172.21.15.87:0/3027584611 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44280c79a0 con 0x7f44280b3d60 2024-09-17T03:14:33.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.531+0000 7f442eb4a640 1 -- 172.21.15.87:0/3027584611 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44280c7f60 con 0x7f44280b3d60 2024-09-17T03:14:33.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.532+0000 7f442eb4a640 1 -- 172.21.15.87:0/3027584611 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4428071e10 con 0x7f44280b3d60 2024-09-17T03:14:33.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.533+0000 7f44257fa640 1 -- 172.21.15.87:0/3027584611 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f441c004420 con 0x7f44280b3d60 2024-09-17T03:14:33.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.533+0000 7f44257fa640 1 --2- 172.21.15.87:0/3027584611 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4414078b90 0x7f441407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:33.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.534+0000 7f44257fa640 1 -- 172.21.15.87:0/3027584611 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f441c09b970 con 0x7f44280b3d60 2024-09-17T03:14:33.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.534+0000 7f442c8e6640 1 --2- 172.21.15.87:0/3027584611 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4414078b90 0x7f441407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:33.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.534+0000 7f442c8e6640 1 --2- 172.21.15.87:0/3027584611 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4414078b90 0x7f441407b0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4418034000 tx=0x7f4418004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:33.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.538+0000 7f44257fa640 1 -- 172.21.15.87:0/3027584611 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f441c063dc0 con 0x7f44280b3d60 2024-09-17T03:14:33.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.746+0000 7f442eb4a640 1 -- 172.21.15.87:0/3027584611 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f44280d2ed0 con 0x7f44280b3d60 2024-09-17T03:14:33.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.747+0000 7f44257fa640 1 -- 172.21.15.87:0/3027584611 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7f441c0636c0 con 0x7f44280b3d60 2024-09-17T03:14:33.746 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T03:14:33.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.749+0000 7f442eb4a640 1 -- 172.21.15.87:0/3027584611 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4414078b90 msgr2=0x7f441407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:33.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.749+0000 7f442eb4a640 1 --2- 172.21.15.87:0/3027584611 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4414078b90 0x7f441407b0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4418034000 tx=0x7f4418004730 comp rx=0 tx=0).stop 2024-09-17T03:14:33.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.749+0000 7f442eb4a640 1 -- 172.21.15.87:0/3027584611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44280b3d60 msgr2=0x7f44280cb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:33.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.749+0000 7f442eb4a640 1 --2- 172.21.15.87:0/3027584611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44280b3d60 0x7f44280cb260 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f441c00ece0 tx=0x7f441c00c6a0 comp rx=0 tx=0).stop 2024-09-17T03:14:33.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.749+0000 7f442eb4a640 1 -- 172.21.15.87:0/3027584611 shutdown_connections 2024-09-17T03:14:33.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.749+0000 7f442eb4a640 1 --2- 172.21.15.87:0/3027584611 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4414078b90 0x7f441407b0a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:33.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.749+0000 7f442eb4a640 1 --2- 172.21.15.87:0/3027584611 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f44280b7f10 0x7f44280a9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:33.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.749+0000 7f442eb4a640 1 --2- 172.21.15.87:0/3027584611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44280b3d60 0x7f44280cb260 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:33.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.749+0000 7f442eb4a640 1 --2- 172.21.15.87:0/3027584611 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44280710e0 0x7f4428159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:33.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.749+0000 7f442eb4a640 1 -- 172.21.15.87:0/3027584611 >> 172.21.15.87:0/3027584611 conn(0x7f4428069330 msgr2=0x7f442815d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:33.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.749+0000 7f442eb4a640 1 -- 172.21.15.87:0/3027584611 shutdown_connections 2024-09-17T03:14:33.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:33.749+0000 7f442eb4a640 1 -- 172.21.15.87:0/3027584611 wait complete. 2024-09-17T03:14:33.758 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T03:14:33.758 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T03:14:33.758 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi149.front.sepia.ceph.com 2024-09-17T03:14:33.759 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-17T03:14:33.759 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-17T03:14:33.759 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T03:14:33.759 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T03:14:33.759 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T03:14:33.759 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-17T03:14:33.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:33.759 DEBUG:teuthology.orchestra.run.smithi149:> ip addr 2024-09-17T03:14:33.778 INFO:teuthology.orchestra.run.smithi149.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:14:33.778 INFO:teuthology.orchestra.run.smithi149.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:14:33.778 INFO:teuthology.orchestra.run.smithi149.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:14:33.779 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:33.779 INFO:teuthology.orchestra.run.smithi149.stdout: inet6 ::1/128 scope host 2024-09-17T03:14:33.779 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:33.779 INFO:teuthology.orchestra.run.smithi149.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:14:33.779 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0c:c4:7a:da:6b:a4 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:14:33.779 INFO:teuthology.orchestra.run.smithi149.stdout: altname enp7s0f0 2024-09-17T03:14:33.779 INFO:teuthology.orchestra.run.smithi149.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:14:33.779 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0c:c4:7a:da:6b:a5 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:14:33.779 INFO:teuthology.orchestra.run.smithi149.stdout: altname enp7s0f1 2024-09-17T03:14:33.779 INFO:teuthology.orchestra.run.smithi149.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:14:33.779 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0c:c4:7a:88:7d:56 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:14:33.779 INFO:teuthology.orchestra.run.smithi149.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T03:14:33.779 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0c:c4:7a:88:7d:57 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:14:33.780 INFO:teuthology.orchestra.run.smithi149.stdout: inet 172.21.15.149/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T03:14:33.780 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft 42775sec preferred_lft 42775sec 2024-09-17T03:14:33.780 INFO:teuthology.orchestra.run.smithi149.stdout: inet6 fe80::ec4:7aff:fe88:7d57/64 scope link noprefixroute 2024-09-17T03:14:33.780 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:33.780 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-17T03:14:33.780 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:14:33.780 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-09-17T03:14:33.780 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link add name ceph-brx type bridge 2024-09-17T03:14:33.780 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip addr flush dev ceph-brx 2024-09-17T03:14:33.780 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link set ceph-brx up 2024-09-17T03:14:33.780 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-17T03:14:33.781 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-09-17T03:14:33.873 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:14:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:14:34.004 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:14:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:14:34.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:34.011 DEBUG:teuthology.orchestra.run.smithi149:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T03:14:34.037 INFO:teuthology.orchestra.run.smithi149.stdout:1 2024-09-17T03:14:34.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:34.038 DEBUG:teuthology.orchestra.run.smithi149:> route 2024-09-17T03:14:34.102 INFO:teuthology.orchestra.run.smithi149.stdout:Kernel IP routing table 2024-09-17T03:14:34.103 INFO:teuthology.orchestra.run.smithi149.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T03:14:34.103 INFO:teuthology.orchestra.run.smithi149.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T03:14:34.103 INFO:teuthology.orchestra.run.smithi149.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T03:14:34.103 INFO:teuthology.orchestra.run.smithi149.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T03:14:34.103 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:14:34.103 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-09-17T03:14:34.103 DEBUG:teuthology.orchestra.run.smithi149:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T03:14:34.103 DEBUG:teuthology.orchestra.run.smithi149:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T03:14:34.103 DEBUG:teuthology.orchestra.run.smithi149:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T03:14:34.103 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-09-17T03:14:34.190 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:14:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:14:34.367 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:14:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:14:34.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:34.374 DEBUG:teuthology.orchestra.run.smithi149:> ip netns list 2024-09-17T03:14:34.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:34.389 DEBUG:teuthology.orchestra.run.smithi149:> ip netns list-id 2024-09-17T03:14:34.445 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:14:34.445 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-09-17T03:14:34.445 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T03:14:34.445 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-17T03:14:34.446 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-09-17T03:14:34.536 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:14:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:14:34.560 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:14:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:14:34.568 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-17T03:14:34.568 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:14:34.569 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-09-17T03:14:34.569 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-17T03:14:34.569 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-17T03:14:34.569 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-17T03:14:34.569 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-17T03:14:34.569 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-17T03:14:34.569 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-09-17T03:14:34.618 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:14:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:14:34.700 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:14:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:14:34.706 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:14:34.707 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-09-17T03:14:34.707 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link set brx.0 up 2024-09-17T03:14:34.707 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link set dev brx.0 master ceph-brx 2024-09-17T03:14:34.707 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-09-17T03:14:34.756 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:14:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:14:34.779 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:14:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:14:34.785 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-17T03:14:34.785 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:14:34.785 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-17T03:14:34.801 INFO:teuthology.orchestra.run.smithi149.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-17T03:14:34.801 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:14:34.801 DEBUG:teuthology.orchestra.run.smithi149:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-17T03:14:34.856 DEBUG:teuthology.orchestra.run.smithi149:> sudo modprobe fuse 2024-09-17T03:14:34.923 DEBUG:teuthology.orchestra.run.smithi149:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T03:14:34.979 INFO:teuthology.orchestra.run.smithi149.stdout:/proc 2024-09-17T03:14:34.980 INFO:teuthology.orchestra.run.smithi149.stdout:/sys 2024-09-17T03:14:34.980 INFO:teuthology.orchestra.run.smithi149.stdout:/dev 2024-09-17T03:14:34.980 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/security 2024-09-17T03:14:34.980 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/shm 2024-09-17T03:14:34.980 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/pts 2024-09-17T03:14:34.980 INFO:teuthology.orchestra.run.smithi149.stdout:/run 2024-09-17T03:14:34.980 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/cgroup 2024-09-17T03:14:34.980 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/pstore 2024-09-17T03:14:34.980 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/bpf 2024-09-17T03:14:34.980 INFO:teuthology.orchestra.run.smithi149.stdout:/ 2024-09-17T03:14:34.980 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/selinux 2024-09-17T03:14:34.980 INFO:teuthology.orchestra.run.smithi149.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T03:14:34.981 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/hugepages 2024-09-17T03:14:34.981 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/mqueue 2024-09-17T03:14:34.981 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/debug 2024-09-17T03:14:34.981 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/tracing 2024-09-17T03:14:34.981 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/fuse/connections 2024-09-17T03:14:34.981 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/config 2024-09-17T03:14:34.981 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T03:14:34.981 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T03:14:34.981 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T03:14:34.981 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T03:14:34.981 INFO:teuthology.orchestra.run.smithi149.stdout:/run/user/1000 2024-09-17T03:14:34.981 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph 2024-09-17T03:14:34.981 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-4 2024-09-17T03:14:34.982 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-5 2024-09-17T03:14:34.982 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-6 2024-09-17T03:14:34.982 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-7 2024-09-17T03:14:34.982 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns 2024-09-17T03:14:34.982 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T03:14:34.982 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T03:14:34.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:34.982 DEBUG:teuthology.orchestra.run.smithi149:> ls /sys/fs/fuse/connections 2024-09-17T03:14:35.035 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T03:14:35.035 DEBUG:teuthology.orchestra.run.smithi149:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-17T03:14:35.077 DEBUG:teuthology.orchestra.run.smithi149:> sudo modprobe fuse 2024-09-17T03:14:35.102 DEBUG:teuthology.orchestra.run.smithi149:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T03:14:35.160 INFO:teuthology.orchestra.run.smithi149.stdout:/proc 2024-09-17T03:14:35.160 INFO:teuthology.orchestra.run.smithi149.stdout:/sys 2024-09-17T03:14:35.160 INFO:teuthology.orchestra.run.smithi149.stdout:/dev 2024-09-17T03:14:35.160 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/security 2024-09-17T03:14:35.160 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/shm 2024-09-17T03:14:35.161 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/pts 2024-09-17T03:14:35.161 INFO:teuthology.orchestra.run.smithi149.stdout:/run 2024-09-17T03:14:35.161 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/cgroup 2024-09-17T03:14:35.161 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/pstore 2024-09-17T03:14:35.161 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/bpf 2024-09-17T03:14:35.161 INFO:teuthology.orchestra.run.smithi149.stdout:/ 2024-09-17T03:14:35.161 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/selinux 2024-09-17T03:14:35.161 INFO:teuthology.orchestra.run.smithi149.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T03:14:35.161 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/hugepages 2024-09-17T03:14:35.161 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/mqueue 2024-09-17T03:14:35.161 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/debug 2024-09-17T03:14:35.161 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/tracing 2024-09-17T03:14:35.161 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/fuse/connections 2024-09-17T03:14:35.162 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/config 2024-09-17T03:14:35.162 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T03:14:35.162 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T03:14:35.162 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T03:14:35.162 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T03:14:35.162 INFO:teuthology.orchestra.run.smithi149.stdout:/run/user/1000 2024-09-17T03:14:35.162 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph 2024-09-17T03:14:35.162 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-4 2024-09-17T03:14:35.163 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-5 2024-09-17T03:14:35.163 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-6 2024-09-17T03:14:35.164 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-7 2024-09-17T03:14:35.164 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns 2024-09-17T03:14:35.164 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T03:14:35.164 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T03:14:35.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:35.165 DEBUG:teuthology.orchestra.run.smithi149:> ls /sys/fs/fuse/connections 2024-09-17T03:14:35.177 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi149.stderr:2024-09-17T03:14:35.177+0000 7f3a01a19480 -1 init, newargv = 0x7f39e4004360 newargc=15 2024-09-17T03:14:35.177 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi149.stderr:ceph-fuse[42372]: starting ceph client 2024-09-17T03:14:35.182 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi149.stderr:ceph-fuse[42372]: starting fuse 2024-09-17T03:14:35.193 INFO:teuthology.orchestra.run.smithi149.stdout:41 2024-09-17T03:14:35.193 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-09-17T03:14:35.193 DEBUG:teuthology.orchestra.run.smithi149:> sudo stdin-killer -- python3 -c ' 2024-09-17T03:14:35.193 DEBUG:teuthology.orchestra.run.smithi149:> import glob 2024-09-17T03:14:35.193 DEBUG:teuthology.orchestra.run.smithi149:> import re 2024-09-17T03:14:35.193 DEBUG:teuthology.orchestra.run.smithi149:> import os 2024-09-17T03:14:35.193 DEBUG:teuthology.orchestra.run.smithi149:> import subprocess 2024-09-17T03:14:35.193 DEBUG:teuthology.orchestra.run.smithi149:> 2024-09-17T03:14:35.194 DEBUG:teuthology.orchestra.run.smithi149:> def _find_admin_socket(client_name): 2024-09-17T03:14:35.194 DEBUG:teuthology.orchestra.run.smithi149:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-17T03:14:35.194 DEBUG:teuthology.orchestra.run.smithi149:> files = glob.glob(asok_path) 2024-09-17T03:14:35.194 DEBUG:teuthology.orchestra.run.smithi149:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-17T03:14:35.194 DEBUG:teuthology.orchestra.run.smithi149:> 2024-09-17T03:14:35.194 DEBUG:teuthology.orchestra.run.smithi149:> # Given a non-glob path, it better be there 2024-09-17T03:14:35.194 DEBUG:teuthology.orchestra.run.smithi149:> if "*" not in asok_path: 2024-09-17T03:14:35.194 DEBUG:teuthology.orchestra.run.smithi149:> assert(len(files) == 1) 2024-09-17T03:14:35.194 DEBUG:teuthology.orchestra.run.smithi149:> return files[0] 2024-09-17T03:14:35.194 DEBUG:teuthology.orchestra.run.smithi149:> 2024-09-17T03:14:35.194 DEBUG:teuthology.orchestra.run.smithi149:> for f in files: 2024-09-17T03:14:35.194 DEBUG:teuthology.orchestra.run.smithi149:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T03:14:35.194 DEBUG:teuthology.orchestra.run.smithi149:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T03:14:35.194 DEBUG:teuthology.orchestra.run.smithi149:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T03:14:35.194 DEBUG:teuthology.orchestra.run.smithi149:> contents = proc_f.read() 2024-09-17T03:14:35.195 DEBUG:teuthology.orchestra.run.smithi149:> if mountpoint in contents: 2024-09-17T03:14:35.195 DEBUG:teuthology.orchestra.run.smithi149:> return f 2024-09-17T03:14:35.195 DEBUG:teuthology.orchestra.run.smithi149:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T03:14:35.195 DEBUG:teuthology.orchestra.run.smithi149:> 2024-09-17T03:14:35.195 DEBUG:teuthology.orchestra.run.smithi149:> print(_find_admin_socket("client.1")) 2024-09-17T03:14:35.195 DEBUG:teuthology.orchestra.run.smithi149:> ' 2024-09-17T03:14:35.312 INFO:teuthology.orchestra.run.smithi149.stdout:/var/run/ceph/ceph-client.1.42372.asok 2024-09-17T03:14:35.315 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:14:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:14:35.322 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.42372.asok 2024-09-17T03:14:35.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:35.323 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.42372.asok status 2024-09-17T03:14:35.466 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-09-17T03:14:35.466 INFO:teuthology.orchestra.run.smithi149.stdout: "metadata": { 2024-09-17T03:14:35.466 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T03:14:35.466 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T03:14:35.466 INFO:teuthology.orchestra.run.smithi149.stdout: "entity_id": "1", 2024-09-17T03:14:35.466 INFO:teuthology.orchestra.run.smithi149.stdout: "hostname": "smithi149", 2024-09-17T03:14:35.466 INFO:teuthology.orchestra.run.smithi149.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-17T03:14:35.466 INFO:teuthology.orchestra.run.smithi149.stdout: "pid": "42372", 2024-09-17T03:14:35.467 INFO:teuthology.orchestra.run.smithi149.stdout: "root": "/" 2024-09-17T03:14:35.467 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-17T03:14:35.467 INFO:teuthology.orchestra.run.smithi149.stdout: "dentry_count": 0, 2024-09-17T03:14:35.467 INFO:teuthology.orchestra.run.smithi149.stdout: "dentry_pinned_count": 0, 2024-09-17T03:14:35.467 INFO:teuthology.orchestra.run.smithi149.stdout: "id": 4660, 2024-09-17T03:14:35.467 INFO:teuthology.orchestra.run.smithi149.stdout: "inst": { 2024-09-17T03:14:35.467 INFO:teuthology.orchestra.run.smithi149.stdout: "name": { 2024-09-17T03:14:35.467 INFO:teuthology.orchestra.run.smithi149.stdout: "type": "client", 2024-09-17T03:14:35.467 INFO:teuthology.orchestra.run.smithi149.stdout: "num": 4660 2024-09-17T03:14:35.467 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-17T03:14:35.467 INFO:teuthology.orchestra.run.smithi149.stdout: "addr": { 2024-09-17T03:14:35.467 INFO:teuthology.orchestra.run.smithi149.stdout: "type": "v1", 2024-09-17T03:14:35.467 INFO:teuthology.orchestra.run.smithi149.stdout: "addr": "192.168.0.1:0", 2024-09-17T03:14:35.467 INFO:teuthology.orchestra.run.smithi149.stdout: "nonce": 904566259 2024-09-17T03:14:35.468 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-17T03:14:35.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-17T03:14:35.468 INFO:teuthology.orchestra.run.smithi149.stdout: "addr": { 2024-09-17T03:14:35.468 INFO:teuthology.orchestra.run.smithi149.stdout: "type": "v1", 2024-09-17T03:14:35.468 INFO:teuthology.orchestra.run.smithi149.stdout: "addr": "192.168.0.1:0", 2024-09-17T03:14:35.468 INFO:teuthology.orchestra.run.smithi149.stdout: "nonce": 904566259 2024-09-17T03:14:35.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-17T03:14:35.468 INFO:teuthology.orchestra.run.smithi149.stdout: "inst_str": "client.4660 192.168.0.1:0/904566259", 2024-09-17T03:14:35.469 INFO:teuthology.orchestra.run.smithi149.stdout: "addr_str": "192.168.0.1:0/904566259", 2024-09-17T03:14:35.469 INFO:teuthology.orchestra.run.smithi149.stdout: "inode_count": 1, 2024-09-17T03:14:35.469 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_epoch": 6, 2024-09-17T03:14:35.469 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_epoch": 23, 2024-09-17T03:14:35.469 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_epoch_barrier": 0, 2024-09-17T03:14:35.469 INFO:teuthology.orchestra.run.smithi149.stdout: "blocklisted": false, 2024-09-17T03:14:35.470 INFO:teuthology.orchestra.run.smithi149.stdout: "fs_name": "cephfs" 2024-09-17T03:14:35.470 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-09-17T03:14:35.475 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-17T03:14:35.475 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T03:14:35.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.602+0000 7ff0e598b640 1 -- 172.21.15.87:0/260012479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0e00cf020 msgr2=0x7ff0e00b5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:35.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.602+0000 7ff0e598b640 1 --2- 172.21.15.87:0/260012479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0e00cf020 0x7ff0e00b5ea0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8005fd0 tx=0x7ff0c802a5d0 comp rx=0 tx=0).stop 2024-09-17T03:14:35.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.603+0000 7ff0e598b640 1 -- 172.21.15.87:0/260012479 shutdown_connections 2024-09-17T03:14:35.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.603+0000 7ff0e598b640 1 --2- 172.21.15.87:0/260012479 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0e00cf020 0x7ff0e00b5ea0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:35.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.603+0000 7ff0e598b640 1 --2- 172.21.15.87:0/260012479 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff0e015c5f0 0x7ff0e015f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:35.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.603+0000 7ff0e598b640 1 -- 172.21.15.87:0/260012479 >> 172.21.15.87:0/260012479 conn(0x7ff0e0069900 msgr2=0x7ff0e00d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:35.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.603+0000 7ff0e598b640 1 -- 172.21.15.87:0/260012479 shutdown_connections 2024-09-17T03:14:35.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.603+0000 7ff0e598b640 1 -- 172.21.15.87:0/260012479 wait complete. 2024-09-17T03:14:35.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.604+0000 7ff0e598b640 1 Processor -- start 2024-09-17T03:14:35.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.604+0000 7ff0e598b640 1 -- start start 2024-09-17T03:14:35.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.606+0000 7ff0e598b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0e00cf020 0x7ff0e00ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:35.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.606+0000 7ff0e598b640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff0e00cf430 0x7ff0e01f6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:35.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.606+0000 7ff0e598b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0e015c5f0 0x7ff0e01560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:35.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.606+0000 7ff0e598b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e00b6770 con 0x7ff0e00cf020 2024-09-17T03:14:35.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.606+0000 7ff0e598b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e00b65f0 con 0x7ff0e015c5f0 2024-09-17T03:14:35.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.606+0000 7ff0e598b640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff0e01627b0 con 0x7ff0e00cf430 2024-09-17T03:14:35.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.606+0000 7ff0deffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0e00cf020 0x7ff0e00ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:35.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.606+0000 7ff0deffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0e00cf020 0x7ff0e00ca8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58116/0 (socket says 172.21.15.87:58116) 2024-09-17T03:14:35.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.606+0000 7ff0deffd640 1 -- 172.21.15.87:0/3131704761 learned_addr learned my addr 172.21.15.87:0/3131704761 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:35.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.606+0000 7ff0d7fff640 1 --2- 172.21.15.87:0/3131704761 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff0e00cf430 0x7ff0e01f6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:35.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.606+0000 7ff0df7fe640 1 --2- 172.21.15.87:0/3131704761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0e015c5f0 0x7ff0e01560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:35.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.606+0000 7ff0deffd640 1 -- 172.21.15.87:0/3131704761 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff0e00cf430 msgr2=0x7ff0e01f6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:35.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.606+0000 7ff0deffd640 1 --2- 172.21.15.87:0/3131704761 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff0e00cf430 0x7ff0e01f6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:35.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.606+0000 7ff0deffd640 1 -- 172.21.15.87:0/3131704761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0e015c5f0 msgr2=0x7ff0e01560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:35.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.606+0000 7ff0deffd640 1 --2- 172.21.15.87:0/3131704761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0e015c5f0 0x7ff0e01560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:35.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.606+0000 7ff0deffd640 1 -- 172.21.15.87:0/3131704761 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e0158820 con 0x7ff0e00cf020 2024-09-17T03:14:35.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.606+0000 7ff0deffd640 1 --2- 172.21.15.87:0/3131704761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0e00cf020 0x7ff0e00ca8d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff0d000d700 tx=0x7ff0d000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:35.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.606+0000 7ff0dcff9640 1 -- 172.21.15.87:0/3131704761 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0017070 con 0x7ff0e00cf020 2024-09-17T03:14:35.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.606+0000 7ff0dcff9640 1 -- 172.21.15.87:0/3131704761 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0d0005310 con 0x7ff0e00cf020 2024-09-17T03:14:35.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.606+0000 7ff0e598b640 1 -- 172.21.15.87:0/3131704761 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0e0158cc0 con 0x7ff0e00cf020 2024-09-17T03:14:35.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.606+0000 7ff0dcff9640 1 -- 172.21.15.87:0/3131704761 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0009820 con 0x7ff0e00cf020 2024-09-17T03:14:35.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.606+0000 7ff0e598b640 1 -- 172.21.15.87:0/3131704761 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0e0159280 con 0x7ff0e00cf020 2024-09-17T03:14:35.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.607+0000 7ff0e598b640 1 -- 172.21.15.87:0/3131704761 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0e00d3970 con 0x7ff0e00cf020 2024-09-17T03:14:35.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.608+0000 7ff0dcff9640 1 -- 172.21.15.87:0/3131704761 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff0d00099c0 con 0x7ff0e00cf020 2024-09-17T03:14:35.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.609+0000 7ff0dcff9640 1 --2- 172.21.15.87:0/3131704761 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff0c0078b90 0x7ff0c007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:35.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.609+0000 7ff0dcff9640 1 -- 172.21.15.87:0/3131704761 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff0d009ad40 con 0x7ff0e00cf020 2024-09-17T03:14:35.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.609+0000 7ff0d7fff640 1 --2- 172.21.15.87:0/3131704761 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff0c0078b90 0x7ff0c007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:35.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.609+0000 7ff0d7fff640 1 --2- 172.21.15.87:0/3131704761 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff0c0078b90 0x7ff0c007b0a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8033000 tx=0x7ff0c8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:35.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.613+0000 7ff0dcff9640 1 -- 172.21.15.87:0/3131704761 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff0d0062bc0 con 0x7ff0e00cf020 2024-09-17T03:14:35.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.822+0000 7ff0e598b640 1 -- 172.21.15.87:0/3131704761 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff0e00b6ad0 con 0x7ff0e00cf020 2024-09-17T03:14:35.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.823+0000 7ff0dcff9640 1 -- 172.21.15.87:0/3131704761 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+73 (secure 0 0 0) 0x7ff0d00675f0 con 0x7ff0e00cf020 2024-09-17T03:14:35.822 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T03:14:35.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.825+0000 7ff0e598b640 1 -- 172.21.15.87:0/3131704761 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff0c0078b90 msgr2=0x7ff0c007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:35.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.825+0000 7ff0e598b640 1 --2- 172.21.15.87:0/3131704761 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff0c0078b90 0x7ff0c007b0a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8033000 tx=0x7ff0c8004730 comp rx=0 tx=0).stop 2024-09-17T03:14:35.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.825+0000 7ff0e598b640 1 -- 172.21.15.87:0/3131704761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0e00cf020 msgr2=0x7ff0e00ca8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:35.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.825+0000 7ff0e598b640 1 --2- 172.21.15.87:0/3131704761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0e00cf020 0x7ff0e00ca8d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff0d000d700 tx=0x7ff0d000dbc0 comp rx=0 tx=0).stop 2024-09-17T03:14:35.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.825+0000 7ff0e598b640 1 -- 172.21.15.87:0/3131704761 shutdown_connections 2024-09-17T03:14:35.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.825+0000 7ff0e598b640 1 --2- 172.21.15.87:0/3131704761 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff0c0078b90 0x7ff0c007b0a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:35.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.825+0000 7ff0e598b640 1 --2- 172.21.15.87:0/3131704761 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0e015c5f0 0x7ff0e01560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:35.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.825+0000 7ff0e598b640 1 --2- 172.21.15.87:0/3131704761 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff0e00cf430 0x7ff0e01f6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:35.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.825+0000 7ff0e598b640 1 --2- 172.21.15.87:0/3131704761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0e00cf020 0x7ff0e00ca8d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:35.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.825+0000 7ff0e598b640 1 -- 172.21.15.87:0/3131704761 >> 172.21.15.87:0/3131704761 conn(0x7ff0e0069900 msgr2=0x7ff0e015da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:35.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.825+0000 7ff0e598b640 1 -- 172.21.15.87:0/3131704761 shutdown_connections 2024-09-17T03:14:35.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:35.825+0000 7ff0e598b640 1 -- 172.21.15.87:0/3131704761 wait complete. 2024-09-17T03:14:35.835 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T03:14:35.836 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T03:14:35.836 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi149.front.sepia.ceph.com 2024-09-17T03:14:35.836 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-17T03:14:35.836 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-17T03:14:35.836 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T03:14:35.836 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T03:14:35.837 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T03:14:35.837 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-17T03:14:35.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:35.837 DEBUG:teuthology.orchestra.run.smithi149:> ip addr 2024-09-17T03:14:35.854 INFO:teuthology.orchestra.run.smithi149.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:14:35.855 INFO:teuthology.orchestra.run.smithi149.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:14:35.855 INFO:teuthology.orchestra.run.smithi149.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:14:35.855 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:35.855 INFO:teuthology.orchestra.run.smithi149.stdout: inet6 ::1/128 scope host 2024-09-17T03:14:35.855 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:35.855 INFO:teuthology.orchestra.run.smithi149.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:14:35.855 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0c:c4:7a:da:6b:a4 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:14:35.855 INFO:teuthology.orchestra.run.smithi149.stdout: altname enp7s0f0 2024-09-17T03:14:35.855 INFO:teuthology.orchestra.run.smithi149.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:14:35.855 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0c:c4:7a:da:6b:a5 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:14:35.855 INFO:teuthology.orchestra.run.smithi149.stdout: altname enp7s0f1 2024-09-17T03:14:35.855 INFO:teuthology.orchestra.run.smithi149.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:14:35.856 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0c:c4:7a:88:7d:56 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:14:35.856 INFO:teuthology.orchestra.run.smithi149.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T03:14:35.856 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0c:c4:7a:88:7d:57 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:14:35.856 INFO:teuthology.orchestra.run.smithi149.stdout: inet 172.21.15.149/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T03:14:35.856 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft 42773sec preferred_lft 42773sec 2024-09-17T03:14:35.856 INFO:teuthology.orchestra.run.smithi149.stdout: inet6 fe80::ec4:7aff:fe88:7d57/64 scope link noprefixroute 2024-09-17T03:14:35.856 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:35.856 INFO:teuthology.orchestra.run.smithi149.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:14:35.856 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 6e:c5:fb:23:5e:cf brd ff:ff:ff:ff:ff:ff 2024-09-17T03:14:35.857 INFO:teuthology.orchestra.run.smithi149.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T03:14:35.857 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:35.857 INFO:teuthology.orchestra.run.smithi149.stdout: inet6 fe80::34fe:7aff:fea4:ad8e/64 scope link 2024-09-17T03:14:35.857 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:35.857 INFO:teuthology.orchestra.run.smithi149.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:14:35.857 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 6e:c5:fb:23:5e:cf brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T03:14:35.857 INFO:teuthology.orchestra.run.smithi149.stdout: inet6 fe80::6cc5:fbff:fe23:5ecf/64 scope link tentative 2024-09-17T03:14:35.857 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:35.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:35.857 DEBUG:teuthology.orchestra.run.smithi149:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T03:14:35.918 INFO:teuthology.orchestra.run.smithi149.stdout:1 2024-09-17T03:14:35.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:35.920 DEBUG:teuthology.orchestra.run.smithi149:> route 2024-09-17T03:14:35.977 INFO:teuthology.orchestra.run.smithi149.stdout:Kernel IP routing table 2024-09-17T03:14:35.977 INFO:teuthology.orchestra.run.smithi149.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T03:14:35.977 INFO:teuthology.orchestra.run.smithi149.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T03:14:35.977 INFO:teuthology.orchestra.run.smithi149.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T03:14:35.978 INFO:teuthology.orchestra.run.smithi149.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T03:14:35.978 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:14:35.978 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-09-17T03:14:35.978 DEBUG:teuthology.orchestra.run.smithi149:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T03:14:35.978 DEBUG:teuthology.orchestra.run.smithi149:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T03:14:35.978 DEBUG:teuthology.orchestra.run.smithi149:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T03:14:35.978 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-09-17T03:14:36.065 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:14:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:14:36.098 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:14:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:14:36.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:36.104 DEBUG:teuthology.orchestra.run.smithi149:> ip netns list 2024-09-17T03:14:36.120 INFO:teuthology.orchestra.run.smithi149.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T03:14:36.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:36.120 DEBUG:teuthology.orchestra.run.smithi149:> ip netns list-id 2024-09-17T03:14:36.176 INFO:teuthology.orchestra.run.smithi149.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-17T03:14:36.177 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:14:36.177 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-09-17T03:14:36.177 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T03:14:36.177 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-17T03:14:36.177 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-09-17T03:14:36.265 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:14:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:14:36.288 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:14:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:14:36.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:36.294 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-17T03:14:36.319 INFO:teuthology.orchestra.run.smithi149.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:14:36.319 INFO:teuthology.orchestra.run.smithi149.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:14:36.319 INFO:teuthology.orchestra.run.smithi149.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:14:36.319 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:36.320 INFO:teuthology.orchestra.run.smithi149.stdout: inet6 ::1/128 scope host 2024-09-17T03:14:36.320 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:36.320 INFO:teuthology.orchestra.run.smithi149.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:14:36.320 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0e:a2:e7:9c:00:7f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T03:14:36.320 INFO:teuthology.orchestra.run.smithi149.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T03:14:36.320 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:36.320 INFO:teuthology.orchestra.run.smithi149.stdout: inet6 fe80::ca2:e7ff:fe9c:7f/64 scope link tentative 2024-09-17T03:14:36.320 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:36.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:36.321 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-17T03:14:36.386 INFO:teuthology.orchestra.run.smithi149.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:14:36.386 INFO:teuthology.orchestra.run.smithi149.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:14:36.386 INFO:teuthology.orchestra.run.smithi149.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:14:36.386 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:36.386 INFO:teuthology.orchestra.run.smithi149.stdout: inet6 ::1/128 scope host 2024-09-17T03:14:36.386 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:36.387 INFO:teuthology.orchestra.run.smithi149.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:14:36.387 INFO:teuthology.orchestra.run.smithi149.stdout: link/ether 0e:a2:e7:9c:00:7f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-17T03:14:36.387 INFO:teuthology.orchestra.run.smithi149.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-17T03:14:36.387 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:36.387 INFO:teuthology.orchestra.run.smithi149.stdout: inet6 fe80::ca2:e7ff:fe9c:7f/64 scope link tentative 2024-09-17T03:14:36.387 INFO:teuthology.orchestra.run.smithi149.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:14:36.388 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-17T03:14:36.388 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:14:36.388 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-09-17T03:14:36.388 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-17T03:14:36.388 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-17T03:14:36.388 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-17T03:14:36.388 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-17T03:14:36.388 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-17T03:14:36.389 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-09-17T03:14:36.476 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:14:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:14:36.536 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:14:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:14:36.542 DEBUG:teuthology.orchestra.run.smithi149:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:14:36.542 DEBUG:teuthology.orchestra.run.smithi149:> set -e 2024-09-17T03:14:36.542 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link set brx.1 up 2024-09-17T03:14:36.542 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link set dev brx.1 master ceph-brx 2024-09-17T03:14:36.542 DEBUG:teuthology.orchestra.run.smithi149:> ') 2024-09-17T03:14:36.590 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:14:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:14:36.613 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:14:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:14:36.619 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-09-17T03:14:36.619 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:14:36.619 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-17T03:14:36.636 INFO:teuthology.orchestra.run.smithi149.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-17T03:14:36.636 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:14:36.637 DEBUG:teuthology.orchestra.run.smithi149:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-17T03:14:36.692 DEBUG:teuthology.orchestra.run.smithi149:> sudo modprobe fuse 2024-09-17T03:14:36.758 DEBUG:teuthology.orchestra.run.smithi149:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T03:14:36.816 INFO:teuthology.orchestra.run.smithi149.stdout:/proc 2024-09-17T03:14:36.816 INFO:teuthology.orchestra.run.smithi149.stdout:/sys 2024-09-17T03:14:36.816 INFO:teuthology.orchestra.run.smithi149.stdout:/dev 2024-09-17T03:14:36.816 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/security 2024-09-17T03:14:36.816 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/shm 2024-09-17T03:14:36.816 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/pts 2024-09-17T03:14:36.816 INFO:teuthology.orchestra.run.smithi149.stdout:/run 2024-09-17T03:14:36.817 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/cgroup 2024-09-17T03:14:36.817 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/pstore 2024-09-17T03:14:36.817 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/bpf 2024-09-17T03:14:36.817 INFO:teuthology.orchestra.run.smithi149.stdout:/ 2024-09-17T03:14:36.817 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/selinux 2024-09-17T03:14:36.817 INFO:teuthology.orchestra.run.smithi149.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T03:14:36.817 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/hugepages 2024-09-17T03:14:36.817 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/mqueue 2024-09-17T03:14:36.817 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/debug 2024-09-17T03:14:36.817 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/tracing 2024-09-17T03:14:36.817 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/fuse/connections 2024-09-17T03:14:36.817 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/config 2024-09-17T03:14:36.817 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T03:14:36.817 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T03:14:36.818 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T03:14:36.818 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T03:14:36.818 INFO:teuthology.orchestra.run.smithi149.stdout:/run/user/1000 2024-09-17T03:14:36.818 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph 2024-09-17T03:14:36.818 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-4 2024-09-17T03:14:36.818 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-5 2024-09-17T03:14:36.818 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-6 2024-09-17T03:14:36.818 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-7 2024-09-17T03:14:36.818 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns 2024-09-17T03:14:36.818 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T03:14:36.818 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T03:14:36.818 INFO:teuthology.orchestra.run.smithi149.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-17T03:14:36.818 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T03:14:36.818 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T03:14:36.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:36.819 DEBUG:teuthology.orchestra.run.smithi149:> ls /sys/fs/fuse/connections 2024-09-17T03:14:36.872 INFO:teuthology.orchestra.run.smithi149.stdout:41 2024-09-17T03:14:36.872 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2024-09-17T03:14:36.872 DEBUG:teuthology.orchestra.run.smithi149:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-09-17T03:14:36.915 DEBUG:teuthology.orchestra.run.smithi149:> sudo modprobe fuse 2024-09-17T03:14:36.940 DEBUG:teuthology.orchestra.run.smithi149:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T03:14:36.999 INFO:teuthology.orchestra.run.smithi149.stdout:/proc 2024-09-17T03:14:36.999 INFO:teuthology.orchestra.run.smithi149.stdout:/sys 2024-09-17T03:14:37.000 INFO:teuthology.orchestra.run.smithi149.stdout:/dev 2024-09-17T03:14:37.000 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/security 2024-09-17T03:14:37.000 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/shm 2024-09-17T03:14:37.000 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/pts 2024-09-17T03:14:37.000 INFO:teuthology.orchestra.run.smithi149.stdout:/run 2024-09-17T03:14:37.000 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/cgroup 2024-09-17T03:14:37.000 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/pstore 2024-09-17T03:14:37.000 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/bpf 2024-09-17T03:14:37.000 INFO:teuthology.orchestra.run.smithi149.stdout:/ 2024-09-17T03:14:37.000 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/selinux 2024-09-17T03:14:37.000 INFO:teuthology.orchestra.run.smithi149.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T03:14:37.000 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/hugepages 2024-09-17T03:14:37.001 INFO:teuthology.orchestra.run.smithi149.stdout:/dev/mqueue 2024-09-17T03:14:37.001 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/debug 2024-09-17T03:14:37.001 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/tracing 2024-09-17T03:14:37.001 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/fs/fuse/connections 2024-09-17T03:14:37.001 INFO:teuthology.orchestra.run.smithi149.stdout:/sys/kernel/config 2024-09-17T03:14:37.001 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T03:14:37.001 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T03:14:37.001 INFO:teuthology.orchestra.run.smithi149.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T03:14:37.001 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T03:14:37.001 INFO:teuthology.orchestra.run.smithi149.stdout:/run/user/1000 2024-09-17T03:14:37.001 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph 2024-09-17T03:14:37.001 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-4 2024-09-17T03:14:37.001 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-5 2024-09-17T03:14:37.002 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-6 2024-09-17T03:14:37.002 INFO:teuthology.orchestra.run.smithi149.stdout:/var/lib/ceph/osd/ceph-7 2024-09-17T03:14:37.002 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns 2024-09-17T03:14:37.002 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T03:14:37.002 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T03:14:37.002 INFO:teuthology.orchestra.run.smithi149.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-17T03:14:37.002 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T03:14:37.002 INFO:teuthology.orchestra.run.smithi149.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T03:14:37.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:37.002 DEBUG:teuthology.orchestra.run.smithi149:> ls /sys/fs/fuse/connections 2024-09-17T03:14:37.008 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi149.stderr:2024-09-17T03:14:37.009+0000 7fee5df31480 -1 init, newargv = 0x7fee40004580 newargc=15 2024-09-17T03:14:37.008 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi149.stderr:ceph-fuse[42829]: starting ceph client 2024-09-17T03:14:37.013 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi149.stderr:ceph-fuse[42829]: starting fuse 2024-09-17T03:14:37.023 INFO:teuthology.orchestra.run.smithi149.stdout:41 2024-09-17T03:14:37.023 INFO:teuthology.orchestra.run.smithi149.stdout:44 2024-09-17T03:14:37.023 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 44] 2024-09-17T03:14:37.024 DEBUG:teuthology.orchestra.run.smithi149:> sudo stdin-killer -- python3 -c ' 2024-09-17T03:14:37.024 DEBUG:teuthology.orchestra.run.smithi149:> import glob 2024-09-17T03:14:37.024 DEBUG:teuthology.orchestra.run.smithi149:> import re 2024-09-17T03:14:37.024 DEBUG:teuthology.orchestra.run.smithi149:> import os 2024-09-17T03:14:37.024 DEBUG:teuthology.orchestra.run.smithi149:> import subprocess 2024-09-17T03:14:37.024 DEBUG:teuthology.orchestra.run.smithi149:> 2024-09-17T03:14:37.024 DEBUG:teuthology.orchestra.run.smithi149:> def _find_admin_socket(client_name): 2024-09-17T03:14:37.024 DEBUG:teuthology.orchestra.run.smithi149:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-09-17T03:14:37.024 DEBUG:teuthology.orchestra.run.smithi149:> files = glob.glob(asok_path) 2024-09-17T03:14:37.024 DEBUG:teuthology.orchestra.run.smithi149:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-09-17T03:14:37.025 DEBUG:teuthology.orchestra.run.smithi149:> 2024-09-17T03:14:37.025 DEBUG:teuthology.orchestra.run.smithi149:> # Given a non-glob path, it better be there 2024-09-17T03:14:37.025 DEBUG:teuthology.orchestra.run.smithi149:> if "*" not in asok_path: 2024-09-17T03:14:37.025 DEBUG:teuthology.orchestra.run.smithi149:> assert(len(files) == 1) 2024-09-17T03:14:37.025 DEBUG:teuthology.orchestra.run.smithi149:> return files[0] 2024-09-17T03:14:37.025 DEBUG:teuthology.orchestra.run.smithi149:> 2024-09-17T03:14:37.025 DEBUG:teuthology.orchestra.run.smithi149:> for f in files: 2024-09-17T03:14:37.025 DEBUG:teuthology.orchestra.run.smithi149:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T03:14:37.025 DEBUG:teuthology.orchestra.run.smithi149:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T03:14:37.025 DEBUG:teuthology.orchestra.run.smithi149:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T03:14:37.025 DEBUG:teuthology.orchestra.run.smithi149:> contents = proc_f.read() 2024-09-17T03:14:37.025 DEBUG:teuthology.orchestra.run.smithi149:> if mountpoint in contents: 2024-09-17T03:14:37.025 DEBUG:teuthology.orchestra.run.smithi149:> return f 2024-09-17T03:14:37.026 DEBUG:teuthology.orchestra.run.smithi149:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T03:14:37.026 DEBUG:teuthology.orchestra.run.smithi149:> 2024-09-17T03:14:37.026 DEBUG:teuthology.orchestra.run.smithi149:> print(_find_admin_socket("client.3")) 2024-09-17T03:14:37.026 DEBUG:teuthology.orchestra.run.smithi149:> ' 2024-09-17T03:14:37.141 INFO:teuthology.orchestra.run.smithi149.stdout:/var/run/ceph/ceph-client.3.42829.asok 2024-09-17T03:14:37.144 INFO:teuthology.orchestra.run.smithi149.stderr:2024-09-17T03:14:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:14:37.151 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.42829.asok 2024-09-17T03:14:37.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:37.152 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.42829.asok status 2024-09-17T03:14:37.293 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-09-17T03:14:37.293 INFO:teuthology.orchestra.run.smithi149.stdout: "metadata": { 2024-09-17T03:14:37.293 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T03:14:37.293 INFO:teuthology.orchestra.run.smithi149.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T03:14:37.293 INFO:teuthology.orchestra.run.smithi149.stdout: "entity_id": "3", 2024-09-17T03:14:37.293 INFO:teuthology.orchestra.run.smithi149.stdout: "hostname": "smithi149", 2024-09-17T03:14:37.293 INFO:teuthology.orchestra.run.smithi149.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-09-17T03:14:37.294 INFO:teuthology.orchestra.run.smithi149.stdout: "pid": "42829", 2024-09-17T03:14:37.294 INFO:teuthology.orchestra.run.smithi149.stdout: "root": "/" 2024-09-17T03:14:37.294 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-17T03:14:37.294 INFO:teuthology.orchestra.run.smithi149.stdout: "dentry_count": 0, 2024-09-17T03:14:37.294 INFO:teuthology.orchestra.run.smithi149.stdout: "dentry_pinned_count": 0, 2024-09-17T03:14:37.294 INFO:teuthology.orchestra.run.smithi149.stdout: "id": 4526, 2024-09-17T03:14:37.294 INFO:teuthology.orchestra.run.smithi149.stdout: "inst": { 2024-09-17T03:14:37.294 INFO:teuthology.orchestra.run.smithi149.stdout: "name": { 2024-09-17T03:14:37.294 INFO:teuthology.orchestra.run.smithi149.stdout: "type": "client", 2024-09-17T03:14:37.294 INFO:teuthology.orchestra.run.smithi149.stdout: "num": 4526 2024-09-17T03:14:37.294 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-17T03:14:37.294 INFO:teuthology.orchestra.run.smithi149.stdout: "addr": { 2024-09-17T03:14:37.294 INFO:teuthology.orchestra.run.smithi149.stdout: "type": "v1", 2024-09-17T03:14:37.295 INFO:teuthology.orchestra.run.smithi149.stdout: "addr": "192.168.0.2:0", 2024-09-17T03:14:37.295 INFO:teuthology.orchestra.run.smithi149.stdout: "nonce": 4141173050 2024-09-17T03:14:37.295 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-09-17T03:14:37.295 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-17T03:14:37.295 INFO:teuthology.orchestra.run.smithi149.stdout: "addr": { 2024-09-17T03:14:37.295 INFO:teuthology.orchestra.run.smithi149.stdout: "type": "v1", 2024-09-17T03:14:37.295 INFO:teuthology.orchestra.run.smithi149.stdout: "addr": "192.168.0.2:0", 2024-09-17T03:14:37.295 INFO:teuthology.orchestra.run.smithi149.stdout: "nonce": 4141173050 2024-09-17T03:14:37.295 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-09-17T03:14:37.295 INFO:teuthology.orchestra.run.smithi149.stdout: "inst_str": "client.4526 192.168.0.2:0/4141173050", 2024-09-17T03:14:37.295 INFO:teuthology.orchestra.run.smithi149.stdout: "addr_str": "192.168.0.2:0/4141173050", 2024-09-17T03:14:37.295 INFO:teuthology.orchestra.run.smithi149.stdout: "inode_count": 1, 2024-09-17T03:14:37.295 INFO:teuthology.orchestra.run.smithi149.stdout: "mds_epoch": 6, 2024-09-17T03:14:37.295 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_epoch": 23, 2024-09-17T03:14:37.296 INFO:teuthology.orchestra.run.smithi149.stdout: "osd_epoch_barrier": 0, 2024-09-17T03:14:37.296 INFO:teuthology.orchestra.run.smithi149.stdout: "blocklisted": false, 2024-09-17T03:14:37.296 INFO:teuthology.orchestra.run.smithi149.stdout: "fs_name": "cephfs" 2024-09-17T03:14:37.296 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-09-17T03:14:37.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:37.302 DEBUG:teuthology.orchestra.run.smithi087:> stat --file-system '--printf=%T 2024-09-17T03:14:37.302 DEBUG:teuthology.orchestra.run.smithi087:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T03:14:37.322 INFO:teuthology.orchestra.run.smithi087.stdout:fuseblk 2024-09-17T03:14:37.322 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T03:14:37.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:37.322 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:14:37.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:37.394 DEBUG:teuthology.orchestra.run.smithi087:> stat --file-system '--printf=%T 2024-09-17T03:14:37.394 DEBUG:teuthology.orchestra.run.smithi087:> ' -- /home/ubuntu/cephtest/mnt.2 2024-09-17T03:14:37.453 INFO:teuthology.orchestra.run.smithi087.stdout:fuseblk 2024-09-17T03:14:37.453 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-09-17T03:14:37.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:37.453 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-17T03:14:37.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:37.525 DEBUG:teuthology.orchestra.run.smithi149:> stat --file-system '--printf=%T 2024-09-17T03:14:37.525 DEBUG:teuthology.orchestra.run.smithi149:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-17T03:14:37.541 INFO:teuthology.orchestra.run.smithi149.stdout:fuseblk 2024-09-17T03:14:37.541 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-17T03:14:37.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:37.541 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-17T03:14:37.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:37.608 DEBUG:teuthology.orchestra.run.smithi149:> stat --file-system '--printf=%T 2024-09-17T03:14:37.608 DEBUG:teuthology.orchestra.run.smithi149:> ' -- /home/ubuntu/cephtest/mnt.3 2024-09-17T03:14:37.664 INFO:teuthology.orchestra.run.smithi149.stdout:fuseblk 2024-09-17T03:14:37.664 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-09-17T03:14:37.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:37.665 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-17T03:14:37.730 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-17T03:14:37.752 INFO:tasks.cephfs_test_runner:Starting test: test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) That a client who does not have the capability for setting 2024-09-17T03:14:37.752 INFO:tasks.cephfs_test_runner:test_forbidden_modification (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-09-17T03:14:37.752 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:14:37.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.876+0000 7f82668ec640 1 -- 172.21.15.87:0/3779367496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8248003a50 msgr2=0x7f8248003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:37.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.876+0000 7f82668ec640 1 --2- 172.21.15.87:0/3779367496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8248003a50 0x7f8248003ec0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f825000c290 tx=0x7f8248005830 comp rx=0 tx=0).stop 2024-09-17T03:14:37.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.876+0000 7f82668ec640 1 -- 172.21.15.87:0/3779367496 shutdown_connections 2024-09-17T03:14:37.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.876+0000 7f82668ec640 1 --2- 172.21.15.87:0/3779367496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8248003a50 0x7f8248003ec0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:37.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.876+0000 7f82668ec640 1 -- 172.21.15.87:0/3779367496 >> 172.21.15.87:0/3779367496 conn(0x7f82600d2230 msgr2=0x7f82600d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:37.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.876+0000 7f82668ec640 1 -- 172.21.15.87:0/3779367496 shutdown_connections 2024-09-17T03:14:37.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.876+0000 7f82668ec640 1 -- 172.21.15.87:0/3779367496 wait complete. 2024-09-17T03:14:37.881 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.878+0000 7f82668ec640 1 Processor -- start 2024-09-17T03:14:37.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.878+0000 7f82668ec640 1 -- start start 2024-09-17T03:14:37.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.879+0000 7f82668ec640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8248003a50 0x7f82601f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:37.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.879+0000 7f82668ec640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82601e3960 0x7f82601e3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:37.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.879+0000 7f82668ec640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f82600b3c60 0x7f82600b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:37.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.879+0000 7f82668ec640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f82600d0f50 con 0x7f8248003a50 2024-09-17T03:14:37.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.879+0000 7f82668ec640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f82600b30d0 con 0x7f82601e3960 2024-09-17T03:14:37.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.879+0000 7f82668ec640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f82600d10d0 con 0x7f82600b3c60 2024-09-17T03:14:37.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.879+0000 7f825ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8248003a50 0x7f82601f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:37.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.879+0000 7f825ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8248003a50 0x7f82601f40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58126/0 (socket says 172.21.15.87:58126) 2024-09-17T03:14:37.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.879+0000 7f825ffff640 1 -- 172.21.15.87:0/422748078 learned_addr learned my addr 172.21.15.87:0/422748078 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:37.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.879+0000 7f825f7fe640 1 --2- 172.21.15.87:0/422748078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82601e3960 0x7f82601e3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:37.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.879+0000 7f8264e89640 1 --2- 172.21.15.87:0/422748078 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f82600b3c60 0x7f82600b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:37.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.879+0000 7f825ffff640 1 -- 172.21.15.87:0/422748078 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f82600b3c60 msgr2=0x7f82600b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:37.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.879+0000 7f825ffff640 1 --2- 172.21.15.87:0/422748078 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f82600b3c60 0x7f82600b4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:37.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.879+0000 7f825ffff640 1 -- 172.21.15.87:0/422748078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82601e3960 msgr2=0x7f82601e3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:37.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.879+0000 7f825ffff640 1 --2- 172.21.15.87:0/422748078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82601e3960 0x7f82601e3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:37.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.879+0000 7f825ffff640 1 -- 172.21.15.87:0/422748078 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82600b45b0 con 0x7f8248003a50 2024-09-17T03:14:37.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.880+0000 7f825f7fe640 1 --2- 172.21.15.87:0/422748078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82601e3960 0x7f82601e3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:37.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.880+0000 7f825ffff640 1 --2- 172.21.15.87:0/422748078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8248003a50 0x7f82601f40e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f825000c3b0 tx=0x7f8250003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:37.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.880+0000 7f825cff9640 1 -- 172.21.15.87:0/422748078 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250001cb0 con 0x7f8248003a50 2024-09-17T03:14:37.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.880+0000 7f825cff9640 1 -- 172.21.15.87:0/422748078 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82500014f0 con 0x7f8248003a50 2024-09-17T03:14:37.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.880+0000 7f82668ec640 1 -- 172.21.15.87:0/422748078 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f826009c1e0 con 0x7f8248003a50 2024-09-17T03:14:37.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.880+0000 7f825cff9640 1 -- 172.21.15.87:0/422748078 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825002e6e0 con 0x7f8248003a50 2024-09-17T03:14:37.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.880+0000 7f82668ec640 1 -- 172.21.15.87:0/422748078 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f826009c7a0 con 0x7f8248003a50 2024-09-17T03:14:37.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.881+0000 7f82668ec640 1 -- 172.21.15.87:0/422748078 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82600cd0f0 con 0x7f8248003a50 2024-09-17T03:14:37.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.881+0000 7f825cff9640 1 -- 172.21.15.87:0/422748078 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8250035020 con 0x7f8248003a50 2024-09-17T03:14:37.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.882+0000 7f825cff9640 1 --2- 172.21.15.87:0/422748078 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f824c078b90 0x7f824c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:37.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.882+0000 7f825cff9640 1 -- 172.21.15.87:0/422748078 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f82500bb030 con 0x7f8248003a50 2024-09-17T03:14:37.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.882+0000 7f825f7fe640 1 --2- 172.21.15.87:0/422748078 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f824c078b90 0x7f824c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:37.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.883+0000 7f825f7fe640 1 --2- 172.21.15.87:0/422748078 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f824c078b90 0x7f824c07b0a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f826009d010 tx=0x7f8254006b70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:37.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:37.887+0000 7f825cff9640 1 -- 172.21.15.87:0/422748078 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8250047070 con 0x7f8248003a50 2024-09-17T03:14:38.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.061+0000 7f82668ec640 1 -- 172.21.15.87:0/422748078 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f82600b1640 con 0x7f8248003a50 2024-09-17T03:14:38.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.062+0000 7f825cff9640 1 -- 172.21.15.87:0/422748078 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16693 (secure 0 0 0) 0x7f82600b45b0 con 0x7f8248003a50 2024-09-17T03:14:38.061 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:38.061 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":23,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:14:15.323716+0000","last_up_change":"2024-09-17T03:14:01.833605+0000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:14:03.433404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:14:11.525547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T03:14:12.626396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6825","nonce":2430409749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6827","nonce":2430409749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6831","nonce":2430409749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6829","nonce":2430409749}]},"public_addr":"172.21.15.87:6825/2430409749","cluster_addr":"172.21.15.87:6827/2430409749","heartbeat_back_addr":"172.21.15.87:6831/2430409749","heartbeat_front_addr":"172.21.15.87:6829/2430409749","state":["exists","up"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6809","nonce":3467605435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6811","nonce":3467605435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6815","nonce":3467605435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6813","nonce":3467605435}]},"public_addr":"172.21.15.87:6809/3467605435","cluster_addr":"172.21.15.87:6811/3467605435","heartbeat_back_addr":"172.21.15.87:6815/3467605435","heartbeat_front_addr":"172.21.15.87:6813/3467605435","state":["exists","up"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6801","nonce":1450834766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6803","nonce":1450834766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6807","nonce":1450834766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6805","nonce":1450834766}]},"public_addr":"172.21.15.87:6801/1450834766","cluster_addr":"172.21.15.87:6803/1450834766","heartbeat_back_addr":"172.21.15.87:6807/1450834766","heartbeat_front_addr":"172.21.15.87:6805/1450834766","state":["exists","up"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6817","nonce":4253161216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6819","nonce":4253161216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6823","nonce":4253161216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6821","nonce":4253161216}]},"public_addr":"172.21.15.87:6817/4253161216","cluster_addr":"172.21.15.87:6819/4253161216","heartbeat_back_addr":"172.21.15.87:6823/4253161216","heartbeat_front_addr":"172.21.15.87:6821/4253161216","state":["exists","up"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6803","nonce":2414907089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6805","nonce":2414907089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6809","nonce":2414907089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6807","nonce":2414907089}]},"public_addr":"172.21.15.149:6803/2414907089","cluster_addr":"172.21.15.149:6805/2414907089","heartbeat_back_addr":"172.21.15.149:6809/2414907089","heartbeat_front_addr":"172.21.15.149:6807/2414907089","state":["exists","up"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6827","nonce":2182999530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6829","nonce":2182999530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6833","nonce":2182999530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6831","nonce":2182999530}]},"public_addr":"172.21.15.149:6827/2182999530","cluster_addr":"172.21.15.149:6829/2182999530","heartbeat_back_addr":"172.21.15.149:6833/2182999530","heartbeat_front_addr":"172.21.15.149:6831/2182999530","state":["exists","up"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6811","nonce":4289519146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6813","nonce":4289519146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6817","nonce":4289519146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6815","nonce":4289519146}]},"public_addr":"172.21.15.149:6811/4289519146","cluster_addr":"172.21.15.149:6813/4289519146","heartbeat_back_addr":"172.21.15.149:6817/4289519146","heartbeat_front_addr":"172.21.15.149:6815/4289519146","state":["exists","up"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6819","nonce":2771010477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6821","nonce":2771010477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6825","nonce":2771010477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6823","nonce":2771010477}]},"public_addr":"172.21.15.149:6819/2771010477","cluster_addr":"172.21.15.149:6821/2771010477","heartbeat_back_addr":"172.21.15.149:6825/2771010477","heartbeat_front_addr":"172.21.15.149:6823/2771010477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.313297+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.642861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.654781+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.302606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.385791+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.915266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.121365+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.394432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:14:38.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.064+0000 7f82668ec640 1 -- 172.21.15.87:0/422748078 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f824c078b90 msgr2=0x7f824c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:38.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.064+0000 7f82668ec640 1 --2- 172.21.15.87:0/422748078 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f824c078b90 0x7f824c07b0a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f826009d010 tx=0x7f8254006b70 comp rx=0 tx=0).stop 2024-09-17T03:14:38.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.064+0000 7f82668ec640 1 -- 172.21.15.87:0/422748078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8248003a50 msgr2=0x7f82601f40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:38.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.064+0000 7f82668ec640 1 --2- 172.21.15.87:0/422748078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8248003a50 0x7f82601f40e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f825000c3b0 tx=0x7f8250003e40 comp rx=0 tx=0).stop 2024-09-17T03:14:38.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.064+0000 7f82668ec640 1 -- 172.21.15.87:0/422748078 shutdown_connections 2024-09-17T03:14:38.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.064+0000 7f82668ec640 1 --2- 172.21.15.87:0/422748078 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f824c078b90 0x7f824c07b0a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:38.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.064+0000 7f82668ec640 1 --2- 172.21.15.87:0/422748078 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f82600b3c60 0x7f82600b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:38.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.064+0000 7f82668ec640 1 --2- 172.21.15.87:0/422748078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f82601e3960 0x7f82601e3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:38.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.064+0000 7f82668ec640 1 --2- 172.21.15.87:0/422748078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8248003a50 0x7f82601f40e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:38.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.064+0000 7f82668ec640 1 -- 172.21.15.87:0/422748078 >> 172.21.15.87:0/422748078 conn(0x7f82600d2230 msgr2=0x7f82600d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:38.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.064+0000 7f82668ec640 1 -- 172.21.15.87:0/422748078 shutdown_connections 2024-09-17T03:14:38.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.064+0000 7f82668ec640 1 -- 172.21.15.87:0/422748078 wait complete. 2024-09-17T03:14:38.075 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T03:14:03.433404+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-17T03:14:11.525547+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-17T03:14:12.626396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T03:14:38.075 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T03:14:38.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.192+0000 7fba7d494640 1 -- 172.21.15.87:0/2953348413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba780d4350 msgr2=0x7fba780d4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:38.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.192+0000 7fba7d494640 1 --2- 172.21.15.87:0/2953348413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba780d4350 0x7fba780d4760 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fba7808e430 tx=0x7fba7002a5f0 comp rx=0 tx=0).stop 2024-09-17T03:14:38.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.192+0000 7fba7d494640 1 -- 172.21.15.87:0/2953348413 shutdown_connections 2024-09-17T03:14:38.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.192+0000 7fba7d494640 1 --2- 172.21.15.87:0/2953348413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba780d4350 0x7fba780d4760 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:38.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.192+0000 7fba7d494640 1 --2- 172.21.15.87:0/2953348413 >> v2:172.21.15.149:3300/0 conn(0x7fba780b3e00 0x7fba780cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:38.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.192+0000 7fba7d494640 1 -- 172.21.15.87:0/2953348413 >> 172.21.15.87:0/2953348413 conn(0x7fba7808bfc0 msgr2=0x7fba7809d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:38.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.192+0000 7fba7d494640 1 -- 172.21.15.87:0/2953348413 shutdown_connections 2024-09-17T03:14:38.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.192+0000 7fba7d494640 1 -- 172.21.15.87:0/2953348413 wait complete. 2024-09-17T03:14:38.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.194+0000 7fba7d494640 1 Processor -- start 2024-09-17T03:14:38.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.194+0000 7fba7d494640 1 -- start start 2024-09-17T03:14:38.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.195+0000 7fba7d494640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fba780b3080 0x7fba78151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:38.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.195+0000 7fba7d494640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba780b3e00 0x7fba78151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:38.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.195+0000 7fba7d494640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba780d4350 0x7fba780cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:38.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.195+0000 7fba7d494640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fba780d29e0 con 0x7fba780b3e00 2024-09-17T03:14:38.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.195+0000 7fba7d494640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fba780d2860 con 0x7fba780d4350 2024-09-17T03:14:38.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.195+0000 7fba7d494640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fba780d2b60 con 0x7fba780b3080 2024-09-17T03:14:38.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.195+0000 7fba767fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba780b3e00 0x7fba78151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:38.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.195+0000 7fba767fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba780b3e00 0x7fba78151e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58136/0 (socket says 172.21.15.87:58136) 2024-09-17T03:14:38.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.195+0000 7fba767fc640 1 -- 172.21.15.87:0/4108180824 learned_addr learned my addr 172.21.15.87:0/4108180824 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:38.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.195+0000 7fba777fe640 1 --2- 172.21.15.87:0/4108180824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba780d4350 0x7fba780cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:38.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.195+0000 7fba76ffd640 1 --2- 172.21.15.87:0/4108180824 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fba780b3080 0x7fba78151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:38.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.196+0000 7fba767fc640 1 -- 172.21.15.87:0/4108180824 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fba780b3080 msgr2=0x7fba78151780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:38.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.196+0000 7fba767fc640 1 --2- 172.21.15.87:0/4108180824 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fba780b3080 0x7fba78151780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:38.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.196+0000 7fba767fc640 1 -- 172.21.15.87:0/4108180824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba780d4350 msgr2=0x7fba780cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:38.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.196+0000 7fba767fc640 1 --2- 172.21.15.87:0/4108180824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba780d4350 0x7fba780cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:38.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.196+0000 7fba767fc640 1 -- 172.21.15.87:0/4108180824 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba780c84f0 con 0x7fba780b3e00 2024-09-17T03:14:38.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.196+0000 7fba767fc640 1 --2- 172.21.15.87:0/4108180824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba780b3e00 0x7fba78151e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00e790 tx=0x7fba6c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:38.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.196+0000 7fba5f7fe640 1 -- 172.21.15.87:0/4108180824 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c00eea0 con 0x7fba780b3e00 2024-09-17T03:14:38.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.196+0000 7fba5f7fe640 1 -- 172.21.15.87:0/4108180824 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba6c00ccc0 con 0x7fba780b3e00 2024-09-17T03:14:38.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.196+0000 7fba7d494640 1 -- 172.21.15.87:0/4108180824 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba780c87d0 con 0x7fba780b3e00 2024-09-17T03:14:38.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.196+0000 7fba5f7fe640 1 -- 172.21.15.87:0/4108180824 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6c005280 con 0x7fba780b3e00 2024-09-17T03:14:38.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.196+0000 7fba7d494640 1 -- 172.21.15.87:0/4108180824 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba78160070 con 0x7fba780b3e00 2024-09-17T03:14:38.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.197+0000 7fba7d494640 1 -- 172.21.15.87:0/4108180824 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba780c8a10 con 0x7fba780b3e00 2024-09-17T03:14:38.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.198+0000 7fba5f7fe640 1 -- 172.21.15.87:0/4108180824 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fba6c005420 con 0x7fba780b3e00 2024-09-17T03:14:38.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.198+0000 7fba5f7fe640 1 --2- 172.21.15.87:0/4108180824 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fba64078b90 0x7fba6407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:38.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.198+0000 7fba5f7fe640 1 -- 172.21.15.87:0/4108180824 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fba6c09b4b0 con 0x7fba780b3e00 2024-09-17T03:14:38.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.199+0000 7fba76ffd640 1 --2- 172.21.15.87:0/4108180824 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fba64078b90 0x7fba6407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:38.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.199+0000 7fba76ffd640 1 --2- 172.21.15.87:0/4108180824 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fba64078b90 0x7fba6407b0a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fba780c7fc0 tx=0x7fba68009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:38.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.203+0000 7fba5f7fe640 1 -- 172.21.15.87:0/4108180824 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fba780c8a10 con 0x7fba780b3e00 2024-09-17T03:14:38.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.382+0000 7fba7d494640 1 -- 172.21.15.87:0/4108180824 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fba780d3080 con 0x7fba780b3e00 2024-09-17T03:14:38.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.382+0000 7fba5f7fe640 1 -- 172.21.15.87:0/4108180824 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7fba6c063330 con 0x7fba780b3e00 2024-09-17T03:14:38.381 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-09-17T03:14:38.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.384+0000 7fba7d494640 1 -- 172.21.15.87:0/4108180824 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fba64078b90 msgr2=0x7fba6407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:38.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.384+0000 7fba7d494640 1 --2- 172.21.15.87:0/4108180824 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fba64078b90 0x7fba6407b0a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fba780c7fc0 tx=0x7fba68009040 comp rx=0 tx=0).stop 2024-09-17T03:14:38.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.384+0000 7fba7d494640 1 -- 172.21.15.87:0/4108180824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba780b3e00 msgr2=0x7fba78151e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:38.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.384+0000 7fba7d494640 1 --2- 172.21.15.87:0/4108180824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba780b3e00 0x7fba78151e10 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00e790 tx=0x7fba6c00ec50 comp rx=0 tx=0).stop 2024-09-17T03:14:38.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.384+0000 7fba7d494640 1 -- 172.21.15.87:0/4108180824 shutdown_connections 2024-09-17T03:14:38.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.384+0000 7fba7d494640 1 --2- 172.21.15.87:0/4108180824 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fba64078b90 0x7fba6407b0a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:38.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.384+0000 7fba7d494640 1 --2- 172.21.15.87:0/4108180824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fba780d4350 0x7fba780cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:38.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.384+0000 7fba7d494640 1 --2- 172.21.15.87:0/4108180824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba780b3e00 0x7fba78151e10 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:38.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.384+0000 7fba7d494640 1 --2- 172.21.15.87:0/4108180824 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fba780b3080 0x7fba78151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:38.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.384+0000 7fba7d494640 1 -- 172.21.15.87:0/4108180824 >> 172.21.15.87:0/4108180824 conn(0x7fba7808bfc0 msgr2=0x7fba780b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:38.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.384+0000 7fba7d494640 1 -- 172.21.15.87:0/4108180824 shutdown_connections 2024-09-17T03:14:38.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.384+0000 7fba7d494640 1 -- 172.21.15.87:0/4108180824 wait complete. 2024-09-17T03:14:38.394 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-17T03:14:38.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.511+0000 7f0c1b0f2640 1 -- 172.21.15.87:0/1935407184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c140ccfe0 msgr2=0x7f0c140cd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:38.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.511+0000 7f0c1b0f2640 1 --2- 172.21.15.87:0/1935407184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c140ccfe0 0x7f0c140cd3f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c009830 tx=0x7f0c0c02a480 comp rx=0 tx=0).stop 2024-09-17T03:14:38.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.511+0000 7f0c1b0f2640 1 -- 172.21.15.87:0/1935407184 shutdown_connections 2024-09-17T03:14:38.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.511+0000 7f0c1b0f2640 1 --2- 172.21.15.87:0/1935407184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c140ccfe0 0x7f0c140cd3f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:38.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.511+0000 7f0c1b0f2640 1 --2- 172.21.15.87:0/1935407184 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c1415b6a0 0x7f0c140d0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:38.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.511+0000 7f0c1b0f2640 1 -- 172.21.15.87:0/1935407184 >> 172.21.15.87:0/1935407184 conn(0x7f0c140cdc80 msgr2=0x7f0c140ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:38.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.511+0000 7f0c1b0f2640 1 -- 172.21.15.87:0/1935407184 shutdown_connections 2024-09-17T03:14:38.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.511+0000 7f0c1b0f2640 1 -- 172.21.15.87:0/1935407184 wait complete. 2024-09-17T03:14:38.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.513+0000 7f0c1b0f2640 1 Processor -- start 2024-09-17T03:14:38.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.513+0000 7f0c1b0f2640 1 -- start start 2024-09-17T03:14:38.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.514+0000 7f0c1b0f2640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c14073e60 0x7f0c1414d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:38.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.514+0000 7f0c1b0f2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c140ccfe0 0x7f0c140c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:38.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.514+0000 7f0c1b0f2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c1415b6a0 0x7f0c140c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:38.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.514+0000 7f0c1b0f2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0c140cbbe0 con 0x7f0c1415b6a0 2024-09-17T03:14:38.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.514+0000 7f0c1b0f2640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0c140cba60 con 0x7f0c140ccfe0 2024-09-17T03:14:38.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.514+0000 7f0c1b0f2640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0c140cbd60 con 0x7f0c14073e60 2024-09-17T03:14:38.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.514+0000 7f0c1968f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c1415b6a0 0x7f0c140c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:38.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.514+0000 7f0c1968f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c1415b6a0 0x7f0c140c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58150/0 (socket says 172.21.15.87:58150) 2024-09-17T03:14:38.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.514+0000 7f0c1968f640 1 -- 172.21.15.87:0/3388913865 learned_addr learned my addr 172.21.15.87:0/3388913865 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:38.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.514+0000 7f0c18e8e640 1 --2- 172.21.15.87:0/3388913865 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c14073e60 0x7f0c1414d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:38.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.514+0000 7f0c13fff640 1 --2- 172.21.15.87:0/3388913865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c140ccfe0 0x7f0c140c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:38.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.515+0000 7f0c1968f640 1 -- 172.21.15.87:0/3388913865 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c14073e60 msgr2=0x7f0c1414d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:38.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.515+0000 7f0c1968f640 1 --2- 172.21.15.87:0/3388913865 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c14073e60 0x7f0c1414d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:38.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.515+0000 7f0c1968f640 1 -- 172.21.15.87:0/3388913865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c140ccfe0 msgr2=0x7f0c140c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:38.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.515+0000 7f0c1968f640 1 --2- 172.21.15.87:0/3388913865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c140ccfe0 0x7f0c140c8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:38.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.515+0000 7f0c1968f640 1 -- 172.21.15.87:0/3388913865 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c141537d0 con 0x7f0c1415b6a0 2024-09-17T03:14:38.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.515+0000 7f0c1968f640 1 --2- 172.21.15.87:0/3388913865 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c1415b6a0 0x7f0c140c9400 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c02aa70 tx=0x7f0c0c033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:38.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.516+0000 7f0c117fa640 1 -- 172.21.15.87:0/3388913865 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c031b50 con 0x7f0c1415b6a0 2024-09-17T03:14:38.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.516+0000 7f0c117fa640 1 -- 172.21.15.87:0/3388913865 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c0c031de0 con 0x7f0c1415b6a0 2024-09-17T03:14:38.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.516+0000 7f0c1b0f2640 1 -- 172.21.15.87:0/3388913865 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c14153ab0 con 0x7f0c1415b6a0 2024-09-17T03:14:38.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.516+0000 7f0c117fa640 1 -- 172.21.15.87:0/3388913865 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c005280 con 0x7f0c1415b6a0 2024-09-17T03:14:38.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.516+0000 7f0c1b0f2640 1 -- 172.21.15.87:0/3388913865 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c140c9ae0 con 0x7f0c1415b6a0 2024-09-17T03:14:38.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.517+0000 7f0c1b0f2640 1 -- 172.21.15.87:0/3388913865 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c140c9d40 con 0x7f0c1415b6a0 2024-09-17T03:14:38.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.518+0000 7f0c117fa640 1 -- 172.21.15.87:0/3388913865 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0c0c005420 con 0x7f0c1415b6a0 2024-09-17T03:14:38.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.518+0000 7f0c117fa640 1 --2- 172.21.15.87:0/3388913865 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0c00078bc0 0x7f0c0007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:38.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.518+0000 7f0c117fa640 1 -- 172.21.15.87:0/3388913865 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f0c0c0bbee0 con 0x7f0c1415b6a0 2024-09-17T03:14:38.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.518+0000 7f0c18e8e640 1 --2- 172.21.15.87:0/3388913865 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0c00078bc0 0x7f0c0007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:38.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.519+0000 7f0c18e8e640 1 --2- 172.21.15.87:0/3388913865 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0c00078bc0 0x7f0c0007b0d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0c04000f60 tx=0x7f0c04009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:38.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.522+0000 7f0c117fa640 1 -- 172.21.15.87:0/3388913865 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0c0c03f030 con 0x7f0c1415b6a0 2024-09-17T03:14:38.693 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.695+0000 7f0c1b0f2640 1 -- 172.21.15.87:0/3388913865 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0c140cd3f0 con 0x7f0c1415b6a0 2024-09-17T03:14:38.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.695+0000 7f0c117fa640 1 -- 172.21.15.87:0/3388913865 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f0c0c031770 con 0x7f0c1415b6a0 2024-09-17T03:14:38.694 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-09-17T03:14:38.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.697+0000 7f0c1b0f2640 1 -- 172.21.15.87:0/3388913865 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0c00078bc0 msgr2=0x7f0c0007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:38.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.697+0000 7f0c1b0f2640 1 --2- 172.21.15.87:0/3388913865 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0c00078bc0 0x7f0c0007b0d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0c04000f60 tx=0x7f0c04009040 comp rx=0 tx=0).stop 2024-09-17T03:14:38.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.697+0000 7f0c1b0f2640 1 -- 172.21.15.87:0/3388913865 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c1415b6a0 msgr2=0x7f0c140c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:38.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.697+0000 7f0c1b0f2640 1 --2- 172.21.15.87:0/3388913865 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c1415b6a0 0x7f0c140c9400 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c02aa70 tx=0x7f0c0c033ba0 comp rx=0 tx=0).stop 2024-09-17T03:14:38.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.697+0000 7f0c1b0f2640 1 -- 172.21.15.87:0/3388913865 shutdown_connections 2024-09-17T03:14:38.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.697+0000 7f0c1b0f2640 1 --2- 172.21.15.87:0/3388913865 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0c00078bc0 0x7f0c0007b0d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:38.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.697+0000 7f0c1b0f2640 1 --2- 172.21.15.87:0/3388913865 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c1415b6a0 0x7f0c140c9400 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:38.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.697+0000 7f0c1b0f2640 1 --2- 172.21.15.87:0/3388913865 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c140ccfe0 0x7f0c140c8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:38.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.697+0000 7f0c1b0f2640 1 --2- 172.21.15.87:0/3388913865 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c14073e60 0x7f0c1414d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:38.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.697+0000 7f0c1b0f2640 1 -- 172.21.15.87:0/3388913865 >> 172.21.15.87:0/3388913865 conn(0x7f0c140cdc80 msgr2=0x7f0c14075c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:38.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.698+0000 7f0c1b0f2640 1 -- 172.21.15.87:0/3388913865 shutdown_connections 2024-09-17T03:14:38.697 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.698+0000 7f0c1b0f2640 1 -- 172.21.15.87:0/3388913865 wait complete. 2024-09-17T03:14:38.707 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-17T03:14:38.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.824+0000 7f1326207640 1 -- 172.21.15.87:0/3645433774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1308004000 msgr2=0x7f1308004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:38.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.824+0000 7f1326207640 1 --2- 172.21.15.87:0/3645433774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1308004000 0x7f1308004800 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f13200b1a20 tx=0x7f1308005830 comp rx=0 tx=0).stop 2024-09-17T03:14:38.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.824+0000 7f131ffff640 1 -- 172.21.15.87:0/3645433774 reap_dead start 2024-09-17T03:14:38.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.824+0000 7f1326207640 1 -- 172.21.15.87:0/3645433774 shutdown_connections 2024-09-17T03:14:38.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.824+0000 7f1326207640 1 -- 172.21.15.87:0/3645433774 >> 172.21.15.87:0/3645433774 conn(0x7f1320069120 msgr2=0x7f1320069530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:38.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.824+0000 7f1326207640 1 -- 172.21.15.87:0/3645433774 shutdown_connections 2024-09-17T03:14:38.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.824+0000 7f1326207640 1 -- 172.21.15.87:0/3645433774 wait complete. 2024-09-17T03:14:38.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.826+0000 7f1326207640 1 Processor -- start 2024-09-17T03:14:38.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.826+0000 7f1326207640 1 -- start start 2024-09-17T03:14:38.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.827+0000 7f1326207640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1308004000 0x7f13200b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:38.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.827+0000 7f1326207640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f132015a630 0x7f13201f6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:38.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.828+0000 7f1326207640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13201f7070 0x7f13201f7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:38.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.828+0000 7f1326207640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f13200d0700 con 0x7f13201f7070 2024-09-17T03:14:38.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.828+0000 7f1326207640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f1320160f70 con 0x7f1308004000 2024-09-17T03:14:38.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.828+0000 7f1326207640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f13200d0880 con 0x7f132015a630 2024-09-17T03:14:38.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.828+0000 7f131ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13201f7070 0x7f13201f7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:38.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.828+0000 7f131ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13201f7070 0x7f13201f7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58170/0 (socket says 172.21.15.87:58170) 2024-09-17T03:14:38.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.828+0000 7f131ffff640 1 -- 172.21.15.87:0/165592416 learned_addr learned my addr 172.21.15.87:0/165592416 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:38.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.828+0000 7f131f7fe640 1 --2- 172.21.15.87:0/165592416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1308004000 0x7f13200b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:38.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.828+0000 7f131effd640 1 --2- 172.21.15.87:0/165592416 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f132015a630 0x7f13201f6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:38.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.828+0000 7f131ffff640 1 -- 172.21.15.87:0/165592416 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f132015a630 msgr2=0x7f13201f6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:38.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.828+0000 7f131ffff640 1 --2- 172.21.15.87:0/165592416 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f132015a630 0x7f13201f6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:38.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.828+0000 7f131ffff640 1 -- 172.21.15.87:0/165592416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1308004000 msgr2=0x7f13200b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:38.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.828+0000 7f131ffff640 1 --2- 172.21.15.87:0/165592416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1308004000 0x7f13200b0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:38.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.828+0000 7f131ffff640 1 -- 172.21.15.87:0/165592416 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13200b1340 con 0x7f13201f7070 2024-09-17T03:14:38.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.828+0000 7f131f7fe640 1 --2- 172.21.15.87:0/165592416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1308004000 0x7f13200b0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:14:38.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.828+0000 7f131ffff640 1 --2- 172.21.15.87:0/165592416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13201f7070 0x7f13201f7480 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f13200ce230 tx=0x7f1318001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:38.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.828+0000 7f1307fff640 1 -- 172.21.15.87:0/165592416 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1318011870 con 0x7f13201f7070 2024-09-17T03:14:38.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.828+0000 7f1307fff640 1 -- 172.21.15.87:0/165592416 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1318017070 con 0x7f13201f7070 2024-09-17T03:14:38.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.828+0000 7f1326207640 1 -- 172.21.15.87:0/165592416 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f132014d7a0 con 0x7f13201f7070 2024-09-17T03:14:38.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.828+0000 7f1307fff640 1 -- 172.21.15.87:0/165592416 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1318011da0 con 0x7f13201f7070 2024-09-17T03:14:38.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.828+0000 7f1326207640 1 -- 172.21.15.87:0/165592416 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f132014dd60 con 0x7f13201f7070 2024-09-17T03:14:38.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.829+0000 7f1326207640 1 -- 172.21.15.87:0/165592416 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13200d0e10 con 0x7f13201f7070 2024-09-17T03:14:38.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.830+0000 7f1307fff640 1 -- 172.21.15.87:0/165592416 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f131801c460 con 0x7f13201f7070 2024-09-17T03:14:38.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.830+0000 7f1307fff640 1 --2- 172.21.15.87:0/165592416 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f130c078bc0 0x7f130c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:38.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.830+0000 7f1307fff640 1 -- 172.21.15.87:0/165592416 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f131809b530 con 0x7f13201f7070 2024-09-17T03:14:38.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.831+0000 7f131f7fe640 1 --2- 172.21.15.87:0/165592416 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f130c078bc0 0x7f130c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:38.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.831+0000 7f131f7fe640 1 --2- 172.21.15.87:0/165592416 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f130c078bc0 0x7f130c07b0d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f13100062f0 tx=0x7f1310001110 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:38.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:38.835+0000 7f1307fff640 1 -- 172.21.15.87:0/165592416 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f13200b1340 con 0x7f13201f7070 2024-09-17T03:14:38.919 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:14:38.923 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:14:39.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.012+0000 7f1326207640 1 -- 172.21.15.87:0/165592416 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f13200b5e10 con 0x7f13201f7070 2024-09-17T03:14:39.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.013+0000 7f1307fff640 1 -- 172.21.15.87:0/165592416 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f1318063a00 con 0x7f13201f7070 2024-09-17T03:14:39.012 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-09-17T03:14:39.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.015+0000 7f1326207640 1 -- 172.21.15.87:0/165592416 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f130c078bc0 msgr2=0x7f130c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:39.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.015+0000 7f1326207640 1 --2- 172.21.15.87:0/165592416 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f130c078bc0 0x7f130c07b0d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f13100062f0 tx=0x7f1310001110 comp rx=0 tx=0).stop 2024-09-17T03:14:39.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.015+0000 7f1326207640 1 -- 172.21.15.87:0/165592416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13201f7070 msgr2=0x7f13201f7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:39.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.015+0000 7f1326207640 1 --2- 172.21.15.87:0/165592416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13201f7070 0x7f13201f7480 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f13200ce230 tx=0x7f1318001a70 comp rx=0 tx=0).stop 2024-09-17T03:14:39.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.015+0000 7f1326207640 1 -- 172.21.15.87:0/165592416 shutdown_connections 2024-09-17T03:14:39.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.015+0000 7f1326207640 1 --2- 172.21.15.87:0/165592416 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f130c078bc0 0x7f130c07b0d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:39.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.015+0000 7f1326207640 1 --2- 172.21.15.87:0/165592416 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13201f7070 0x7f13201f7480 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:39.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.015+0000 7f1326207640 1 --2- 172.21.15.87:0/165592416 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f132015a630 0x7f13201f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:39.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.015+0000 7f1326207640 1 --2- 172.21.15.87:0/165592416 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1308004000 0x7f13200b0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:39.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.015+0000 7f1326207640 1 -- 172.21.15.87:0/165592416 >> 172.21.15.87:0/165592416 conn(0x7f1320069120 msgr2=0x7f13200b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:39.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.015+0000 7f1326207640 1 -- 172.21.15.87:0/165592416 shutdown_connections 2024-09-17T03:14:39.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.015+0000 7f1326207640 1 -- 172.21.15.87:0/165592416 wait complete. 2024-09-17T03:14:39.026 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2024-09-17T03:14:39.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.180+0000 7f6396b13640 1 -- 172.21.15.87:0/2119810349 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63900b3d00 msgr2=0x7f63900b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:39.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.180+0000 7f6396b13640 1 --2- 172.21.15.87:0/2119810349 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63900b3d00 0x7f63900b4110 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6384005fd0 tx=0x7f638402a970 comp rx=0 tx=0).stop 2024-09-17T03:14:39.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.180+0000 7f6396b13640 1 -- 172.21.15.87:0/2119810349 shutdown_connections 2024-09-17T03:14:39.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.180+0000 7f6396b13640 1 --2- 172.21.15.87:0/2119810349 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63900b3d00 0x7f63900b4110 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:39.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.180+0000 7f6396b13640 1 --2- 172.21.15.87:0/2119810349 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f639015c6c0 0x7f6390160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:39.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.180+0000 7f6396b13640 1 -- 172.21.15.87:0/2119810349 >> 172.21.15.87:0/2119810349 conn(0x7f6390069360 msgr2=0x7f63900ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:39.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.180+0000 7f6396b13640 1 -- 172.21.15.87:0/2119810349 shutdown_connections 2024-09-17T03:14:39.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.180+0000 7f6396b13640 1 -- 172.21.15.87:0/2119810349 wait complete. 2024-09-17T03:14:39.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.182+0000 7f6396b13640 1 Processor -- start 2024-09-17T03:14:39.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.182+0000 7f6396b13640 1 -- start start 2024-09-17T03:14:39.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.183+0000 7f6396b13640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63900710e0 0x7f63901547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:39.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.183+0000 7f6396b13640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63900b3d00 0x7f639015f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:39.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.183+0000 7f6396b13640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f639015c6c0 0x7f639015f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:39.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.183+0000 7f6396b13640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f63900d1470 con 0x7f63900b3d00 2024-09-17T03:14:39.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.183+0000 7f6396b13640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f63900b80d0 con 0x7f639015c6c0 2024-09-17T03:14:39.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.183+0000 7f6396b13640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f63900d15f0 con 0x7f63900710e0 2024-09-17T03:14:39.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.183+0000 7f638ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63900b3d00 0x7f639015f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:39.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.184+0000 7f638ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63900b3d00 0x7f639015f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58180/0 (socket says 172.21.15.87:58180) 2024-09-17T03:14:39.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.184+0000 7f638ffff640 1 -- 172.21.15.87:0/3037467443 learned_addr learned my addr 172.21.15.87:0/3037467443 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:39.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.184+0000 7f63950b0640 1 --2- 172.21.15.87:0/3037467443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f639015c6c0 0x7f639015f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:39.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.184+0000 7f63948af640 1 --2- 172.21.15.87:0/3037467443 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63900710e0 0x7f63901547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:39.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.184+0000 7f638ffff640 1 -- 172.21.15.87:0/3037467443 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63900710e0 msgr2=0x7f63901547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:39.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.184+0000 7f638ffff640 1 --2- 172.21.15.87:0/3037467443 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63900710e0 0x7f63901547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:39.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.184+0000 7f638ffff640 1 -- 172.21.15.87:0/3037467443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f639015c6c0 msgr2=0x7f639015f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:39.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.184+0000 7f638ffff640 1 --2- 172.21.15.87:0/3037467443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f639015c6c0 0x7f639015f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:39.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.184+0000 7f638ffff640 1 -- 172.21.15.87:0/3037467443 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f639015de40 con 0x7f63900b3d00 2024-09-17T03:14:39.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.184+0000 7f638ffff640 1 --2- 172.21.15.87:0/3037467443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63900b3d00 0x7f639015f2b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6384006010 tx=0x7f6384003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:39.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.184+0000 7f638d7fa640 1 -- 172.21.15.87:0/3037467443 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6384034040 con 0x7f63900b3d00 2024-09-17T03:14:39.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.184+0000 7f638d7fa640 1 -- 172.21.15.87:0/3037467443 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6384004640 con 0x7f63900b3d00 2024-09-17T03:14:39.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.184+0000 7f638d7fa640 1 -- 172.21.15.87:0/3037467443 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6384045480 con 0x7f63900b3d00 2024-09-17T03:14:39.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.184+0000 7f6396b13640 1 -- 172.21.15.87:0/3037467443 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f639015e120 con 0x7f63900b3d00 2024-09-17T03:14:39.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.184+0000 7f6396b13640 1 -- 172.21.15.87:0/3037467443 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f639015e6e0 con 0x7f63900b3d00 2024-09-17T03:14:39.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.185+0000 7f6396b13640 1 -- 172.21.15.87:0/3037467443 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f639015dbb0 con 0x7f63900b3d00 2024-09-17T03:14:39.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.186+0000 7f638d7fa640 1 -- 172.21.15.87:0/3037467443 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6384037020 con 0x7f63900b3d00 2024-09-17T03:14:39.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.186+0000 7f638d7fa640 1 --2- 172.21.15.87:0/3037467443 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f637c078b90 0x7f637c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:39.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.187+0000 7f638d7fa640 1 -- 172.21.15.87:0/3037467443 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f63840bc5c0 con 0x7f63900b3d00 2024-09-17T03:14:39.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.187+0000 7f63948af640 1 --2- 172.21.15.87:0/3037467443 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f637c078b90 0x7f637c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:39.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.187+0000 7f63948af640 1 --2- 172.21.15.87:0/3037467443 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f637c078b90 0x7f637c07b0a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f63800058c0 tx=0x7f6380005f60 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:39.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.191+0000 7f638d7fa640 1 -- 172.21.15.87:0/3037467443 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f63900b4650 con 0x7f63900b3d00 2024-09-17T03:14:39.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.433+0000 7f6396b13640 1 -- 172.21.15.87:0/3037467443 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) -- 0x7f6390071d00 con 0x7f63900b3d00 2024-09-17T03:14:39.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.518+0000 7f638d7fa640 1 -- 172.21.15.87:0/3037467443 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v45) ==== 150+0+0 (secure 0 0 0) 0x7f63840843b0 con 0x7f63900b3d00 2024-09-17T03:14:39.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.521+0000 7f6396b13640 1 -- 172.21.15.87:0/3037467443 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f637c078b90 msgr2=0x7f637c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:39.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.521+0000 7f6396b13640 1 --2- 172.21.15.87:0/3037467443 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f637c078b90 0x7f637c07b0a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f63800058c0 tx=0x7f6380005f60 comp rx=0 tx=0).stop 2024-09-17T03:14:39.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.521+0000 7f6396b13640 1 -- 172.21.15.87:0/3037467443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63900b3d00 msgr2=0x7f639015f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:39.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.521+0000 7f6396b13640 1 --2- 172.21.15.87:0/3037467443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63900b3d00 0x7f639015f2b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6384006010 tx=0x7f6384003f90 comp rx=0 tx=0).stop 2024-09-17T03:14:39.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.521+0000 7f6396b13640 1 -- 172.21.15.87:0/3037467443 shutdown_connections 2024-09-17T03:14:39.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.521+0000 7f6396b13640 1 --2- 172.21.15.87:0/3037467443 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f637c078b90 0x7f637c07b0a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:39.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.521+0000 7f6396b13640 1 --2- 172.21.15.87:0/3037467443 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f639015c6c0 0x7f639015f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:39.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.521+0000 7f6396b13640 1 --2- 172.21.15.87:0/3037467443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63900b3d00 0x7f639015f2b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:39.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.521+0000 7f6396b13640 1 --2- 172.21.15.87:0/3037467443 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f63900710e0 0x7f63901547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:39.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.521+0000 7f6396b13640 1 -- 172.21.15.87:0/3037467443 >> 172.21.15.87:0/3037467443 conn(0x7f6390069360 msgr2=0x7f63900d2cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:39.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.521+0000 7f6396b13640 1 -- 172.21.15.87:0/3037467443 shutdown_connections 2024-09-17T03:14:39.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.521+0000 7f6396b13640 1 -- 172.21.15.87:0/3037467443 wait complete. 2024-09-17T03:14:39.531 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T03:14:39.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.644+0000 7fdb340cc640 1 -- 172.21.15.87:0/2477589170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb18003a00 msgr2=0x7fdb18003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:39.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.644+0000 7fdb340cc640 1 --2- 172.21.15.87:0/2477589170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb18003a00 0x7fdb18003e70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009b50 tx=0x7fdb18005830 comp rx=0 tx=0).stop 2024-09-17T03:14:39.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.644+0000 7fdb32669640 1 -- 172.21.15.87:0/2477589170 reap_dead start 2024-09-17T03:14:39.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.644+0000 7fdb340cc640 1 -- 172.21.15.87:0/2477589170 shutdown_connections 2024-09-17T03:14:39.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.644+0000 7fdb340cc640 1 -- 172.21.15.87:0/2477589170 >> 172.21.15.87:0/2477589170 conn(0x7fdb2c0d2230 msgr2=0x7fdb2c0d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:39.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.644+0000 7fdb340cc640 1 -- 172.21.15.87:0/2477589170 shutdown_connections 2024-09-17T03:14:39.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.644+0000 7fdb340cc640 1 -- 172.21.15.87:0/2477589170 wait complete. 2024-09-17T03:14:39.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.646+0000 7fdb340cc640 1 Processor -- start 2024-09-17T03:14:39.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.646+0000 7fdb340cc640 1 -- start start 2024-09-17T03:14:39.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.647+0000 7fdb340cc640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdb18003a00 0x7fdb2c0cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:39.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.647+0000 7fdb340cc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb2c1f4530 0x7fdb2c1e7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:39.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.647+0000 7fdb340cc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb2c09c5a0 0x7fdb2c09c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:39.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.647+0000 7fdb340cc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2c0d13d0 con 0x7fdb2c09c5a0 2024-09-17T03:14:39.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.647+0000 7fdb340cc640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2c0d1250 con 0x7fdb2c1f4530 2024-09-17T03:14:39.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.647+0000 7fdb340cc640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fdb2c0cee00 con 0x7fdb18003a00 2024-09-17T03:14:39.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.647+0000 7fdb32669640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb2c09c5a0 0x7fdb2c09c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:39.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.647+0000 7fdb32669640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb2c09c5a0 0x7fdb2c09c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58206/0 (socket says 172.21.15.87:58206) 2024-09-17T03:14:39.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.647+0000 7fdb32669640 1 -- 172.21.15.87:0/3191805897 learned_addr learned my addr 172.21.15.87:0/3191805897 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:39.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.647+0000 7fdb31667640 1 --2- 172.21.15.87:0/3191805897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb2c1f4530 0x7fdb2c1e7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:39.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.647+0000 7fdb31e68640 1 --2- 172.21.15.87:0/3191805897 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdb18003a00 0x7fdb2c0cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:39.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.647+0000 7fdb32669640 1 -- 172.21.15.87:0/3191805897 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdb18003a00 msgr2=0x7fdb2c0cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:39.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.647+0000 7fdb32669640 1 --2- 172.21.15.87:0/3191805897 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdb18003a00 0x7fdb2c0cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:39.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.647+0000 7fdb32669640 1 -- 172.21.15.87:0/3191805897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb2c1f4530 msgr2=0x7fdb2c1e7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:39.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.647+0000 7fdb32669640 1 --2- 172.21.15.87:0/3191805897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb2c1f4530 0x7fdb2c1e7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:39.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.647+0000 7fdb32669640 1 -- 172.21.15.87:0/3191805897 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb2c09cef0 con 0x7fdb2c09c5a0 2024-09-17T03:14:39.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.648+0000 7fdb31e68640 1 --2- 172.21.15.87:0/3191805897 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdb18003a00 0x7fdb2c0cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:14:39.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.648+0000 7fdb32669640 1 --2- 172.21.15.87:0/3191805897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb2c09c5a0 0x7fdb2c09c9b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c074cb0 tx=0x7fdb24010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:39.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.648+0000 7fdb227fc640 1 -- 172.21.15.87:0/3191805897 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb24017070 con 0x7fdb2c09c5a0 2024-09-17T03:14:39.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.648+0000 7fdb227fc640 1 -- 172.21.15.87:0/3191805897 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb24015070 con 0x7fdb2c09c5a0 2024-09-17T03:14:39.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.648+0000 7fdb340cc640 1 -- 172.21.15.87:0/3191805897 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb2c0cdef0 con 0x7fdb2c09c5a0 2024-09-17T03:14:39.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.648+0000 7fdb227fc640 1 -- 172.21.15.87:0/3191805897 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb24004d80 con 0x7fdb2c09c5a0 2024-09-17T03:14:39.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.648+0000 7fdb340cc640 1 -- 172.21.15.87:0/3191805897 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb2c09eb00 con 0x7fdb2c09c5a0 2024-09-17T03:14:39.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.649+0000 7fdb340cc640 1 -- 172.21.15.87:0/3191805897 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb2c1e96e0 con 0x7fdb2c09c5a0 2024-09-17T03:14:39.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.650+0000 7fdb227fc640 1 -- 172.21.15.87:0/3191805897 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdb24011440 con 0x7fdb2c09c5a0 2024-09-17T03:14:39.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.650+0000 7fdb227fc640 1 --2- 172.21.15.87:0/3191805897 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdb10078bc0 0x7fdb1007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:39.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.651+0000 7fdb227fc640 1 -- 172.21.15.87:0/3191805897 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fdb2409ae40 con 0x7fdb2c09c5a0 2024-09-17T03:14:39.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.651+0000 7fdb31e68640 1 --2- 172.21.15.87:0/3191805897 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdb10078bc0 0x7fdb1007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:39.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.651+0000 7fdb31e68640 1 --2- 172.21.15.87:0/3191805897 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdb10078bc0 0x7fdb1007b0d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009b50 tx=0x7fdb2802d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:39.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.655+0000 7fdb227fc640 1 -- 172.21.15.87:0/3191805897 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fdb2c1e96e0 con 0x7fdb2c09c5a0 2024-09-17T03:14:39.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.830+0000 7fdb340cc640 1 -- 172.21.15.87:0/3191805897 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fdb2c0cd390 con 0x7fdb2c09c5a0 2024-09-17T03:14:39.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.830+0000 7fdb227fc640 1 -- 172.21.15.87:0/3191805897 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v31) ==== 59+0+2294 (secure 0 0 0) 0x7fdb2c09cef0 con 0x7fdb2c09c5a0 2024-09-17T03:14:39.829 INFO:teuthology.orchestra.run.smithi087.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T03:14:39.829 INFO:teuthology.orchestra.run.smithi087.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T03:14:39.829 INFO:teuthology.orchestra.run.smithi087.stdout:mgr advanced client_mount_timeout 30 2024-09-17T03:14:39.830 INFO:teuthology.orchestra.run.smithi087.stdout:mgr advanced debug_client 20/20 2024-09-17T03:14:39.830 INFO:teuthology.orchestra.run.smithi087.stdout:mgr advanced debug_mgr 20/20 2024-09-17T03:14:39.830 INFO:teuthology.orchestra.run.smithi087.stdout:mgr advanced debug_ms 1/1 2024-09-17T03:14:39.830 INFO:teuthology.orchestra.run.smithi087.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T03:14:39.830 INFO:teuthology.orchestra.run.smithi087.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T03:14:39.830 INFO:teuthology.orchestra.run.smithi087.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4490.708381 2024-09-17T03:14:39.830 INFO:teuthology.orchestra.run.smithi087.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4822.941320 2024-09-17T03:14:39.830 INFO:teuthology.orchestra.run.smithi087.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 5249.365509 2024-09-17T03:14:39.830 INFO:teuthology.orchestra.run.smithi087.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4538.303788 2024-09-17T03:14:39.830 INFO:teuthology.orchestra.run.smithi087.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6974.722182 2024-09-17T03:14:39.830 INFO:teuthology.orchestra.run.smithi087.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 7605.766974 2024-09-17T03:14:39.830 INFO:teuthology.orchestra.run.smithi087.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5406.650094 2024-09-17T03:14:39.830 INFO:teuthology.orchestra.run.smithi087.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5658.926090 2024-09-17T03:14:39.831 INFO:teuthology.orchestra.run.smithi087.stdout:mds advanced debug_mds 20/20 2024-09-17T03:14:39.831 INFO:teuthology.orchestra.run.smithi087.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T03:14:39.831 INFO:teuthology.orchestra.run.smithi087.stdout:mds advanced debug_ms 2/2 2024-09-17T03:14:39.831 INFO:teuthology.orchestra.run.smithi087.stdout:mds dev mds_debug_frag true 2024-09-17T03:14:39.831 INFO:teuthology.orchestra.run.smithi087.stdout:mds dev mds_debug_scatterstat true 2024-09-17T03:14:39.831 INFO:teuthology.orchestra.run.smithi087.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T03:14:39.831 INFO:teuthology.orchestra.run.smithi087.stdout:mds dev mds_verify_scatter true 2024-09-17T03:14:39.831 INFO:teuthology.orchestra.run.smithi087.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T03:14:39.831 INFO:teuthology.orchestra.run.smithi087.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T03:14:39.831 INFO:teuthology.orchestra.run.smithi087.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T03:14:39.831 INFO:teuthology.orchestra.run.smithi087.stdout:client advanced client_mount_timeout 600 2024-09-17T03:14:39.831 INFO:teuthology.orchestra.run.smithi087.stdout:client advanced debug_client 20/20 2024-09-17T03:14:39.831 INFO:teuthology.orchestra.run.smithi087.stdout:client advanced debug_ms 1/1 2024-09-17T03:14:39.831 INFO:teuthology.orchestra.run.smithi087.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T03:14:39.832 INFO:teuthology.orchestra.run.smithi087.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T03:14:39.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.833+0000 7fdb340cc640 1 -- 172.21.15.87:0/3191805897 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdb10078bc0 msgr2=0x7fdb1007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:39.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.833+0000 7fdb340cc640 1 --2- 172.21.15.87:0/3191805897 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdb10078bc0 0x7fdb1007b0d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009b50 tx=0x7fdb2802d040 comp rx=0 tx=0).stop 2024-09-17T03:14:39.832 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.833+0000 7fdb340cc640 1 -- 172.21.15.87:0/3191805897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb2c09c5a0 msgr2=0x7fdb2c09c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:39.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.833+0000 7fdb340cc640 1 --2- 172.21.15.87:0/3191805897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb2c09c5a0 0x7fdb2c09c9b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c074cb0 tx=0x7fdb24010f40 comp rx=0 tx=0).stop 2024-09-17T03:14:39.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.833+0000 7fdb340cc640 1 -- 172.21.15.87:0/3191805897 shutdown_connections 2024-09-17T03:14:39.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.833+0000 7fdb340cc640 1 --2- 172.21.15.87:0/3191805897 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdb10078bc0 0x7fdb1007b0d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:39.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.833+0000 7fdb340cc640 1 --2- 172.21.15.87:0/3191805897 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb2c09c5a0 0x7fdb2c09c9b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:39.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.833+0000 7fdb340cc640 1 --2- 172.21.15.87:0/3191805897 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb2c1f4530 0x7fdb2c1e7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:39.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.833+0000 7fdb340cc640 1 --2- 172.21.15.87:0/3191805897 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdb18003a00 0x7fdb2c0cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:39.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.833+0000 7fdb340cc640 1 -- 172.21.15.87:0/3191805897 >> 172.21.15.87:0/3191805897 conn(0x7fdb2c0d2230 msgr2=0x7fdb2c0af770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:39.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.833+0000 7fdb340cc640 1 -- 172.21.15.87:0/3191805897 shutdown_connections 2024-09-17T03:14:39.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.833+0000 7fdb340cc640 1 -- 172.21.15.87:0/3191805897 wait complete. 2024-09-17T03:14:39.842 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T03:14:39.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.959+0000 7f42fbf51640 1 -- 172.21.15.87:0/1746649285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42f40b96f0 msgr2=0x7f42f40d2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:39.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.959+0000 7f42fbf51640 1 --2- 172.21.15.87:0/1746649285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42f40b96f0 0x7f42f40d2d90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f42f408ca30 tx=0x7f42e802a560 comp rx=0 tx=0).stop 2024-09-17T03:14:39.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.959+0000 7f42fbf51640 1 -- 172.21.15.87:0/1746649285 shutdown_connections 2024-09-17T03:14:39.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.959+0000 7f42fbf51640 1 --2- 172.21.15.87:0/1746649285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42f40b96f0 0x7f42f40d2d90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:39.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.959+0000 7f42fbf51640 1 --2- 172.21.15.87:0/1746649285 >> v2:172.21.15.149:3300/0 conn(0x7f42f40b9ca0 0x7f42f40b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:39.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.959+0000 7f42fbf51640 1 -- 172.21.15.87:0/1746649285 >> 172.21.15.87:0/1746649285 conn(0x7f42f409d840 msgr2=0x7f42f408bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:39.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.959+0000 7f42fbf51640 1 -- 172.21.15.87:0/1746649285 shutdown_connections 2024-09-17T03:14:39.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.959+0000 7f42fbf51640 1 -- 172.21.15.87:0/1746649285 wait complete. 2024-09-17T03:14:39.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.961+0000 7f42fbf51640 1 Processor -- start 2024-09-17T03:14:39.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.961+0000 7f42fbf51640 1 -- start start 2024-09-17T03:14:39.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.962+0000 7f42fbf51640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42f406e020 0x7f42f415dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:39.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.962+0000 7f42fbf51640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f42f40b96f0 0x7f42f40cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:39.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.962+0000 7f42fbf51640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42f40b9ca0 0x7f42f40cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:39.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.962+0000 7f42fbf51640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f42f40cf1b0 con 0x7f42f40b9ca0 2024-09-17T03:14:39.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.962+0000 7f42fbf51640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f42f40d1710 con 0x7f42f406e020 2024-09-17T03:14:39.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.962+0000 7f42fbf51640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f42f40cf330 con 0x7f42f40b96f0 2024-09-17T03:14:39.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.962+0000 7f42fa4ee640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42f40b9ca0 0x7f42f40cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:39.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.962+0000 7f42fa4ee640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42f40b9ca0 0x7f42f40cc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58230/0 (socket says 172.21.15.87:58230) 2024-09-17T03:14:39.962 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.962+0000 7f42fa4ee640 1 -- 172.21.15.87:0/521707056 learned_addr learned my addr 172.21.15.87:0/521707056 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:39.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.962+0000 7f42f9ced640 1 --2- 172.21.15.87:0/521707056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42f406e020 0x7f42f415dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:39.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.962+0000 7f42f94ec640 1 --2- 172.21.15.87:0/521707056 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f42f40b96f0 0x7f42f40cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:39.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.962+0000 7f42fa4ee640 1 -- 172.21.15.87:0/521707056 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f42f40b96f0 msgr2=0x7f42f40cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:39.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.962+0000 7f42fa4ee640 1 --2- 172.21.15.87:0/521707056 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f42f40b96f0 0x7f42f40cc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:39.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.962+0000 7f42fa4ee640 1 -- 172.21.15.87:0/521707056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42f406e020 msgr2=0x7f42f415dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:39.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.962+0000 7f42fa4ee640 1 --2- 172.21.15.87:0/521707056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42f406e020 0x7f42f415dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:39.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.962+0000 7f42fa4ee640 1 -- 172.21.15.87:0/521707056 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f40ce7c0 con 0x7f42f40b9ca0 2024-09-17T03:14:39.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.962+0000 7f42fa4ee640 1 --2- 172.21.15.87:0/521707056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42f40b9ca0 0x7f42f40cc870 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00c6c0 tx=0x7f42ec00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:39.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.963+0000 7f42e67fc640 1 -- 172.21.15.87:0/521707056 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec003b70 con 0x7f42f40b9ca0 2024-09-17T03:14:39.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.963+0000 7f42e67fc640 1 -- 172.21.15.87:0/521707056 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42ec003d30 con 0x7f42f40b9ca0 2024-09-17T03:14:39.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.963+0000 7f42e67fc640 1 -- 172.21.15.87:0/521707056 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ec005330 con 0x7f42f40b9ca0 2024-09-17T03:14:39.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.963+0000 7f42fbf51640 1 -- 172.21.15.87:0/521707056 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f40cb850 con 0x7f42f40b9ca0 2024-09-17T03:14:39.963 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.963+0000 7f42fbf51640 1 -- 172.21.15.87:0/521707056 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f40cbe10 con 0x7f42f40b9ca0 2024-09-17T03:14:39.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.964+0000 7f42fbf51640 1 -- 172.21.15.87:0/521707056 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42f40ce530 con 0x7f42f40b9ca0 2024-09-17T03:14:39.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.964+0000 7f42e67fc640 1 -- 172.21.15.87:0/521707056 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f42ec0054d0 con 0x7f42f40b9ca0 2024-09-17T03:14:39.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.965+0000 7f42e67fc640 1 --2- 172.21.15.87:0/521707056 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f42d8078b90 0x7f42d807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:39.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.965+0000 7f42e67fc640 1 -- 172.21.15.87:0/521707056 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f42ec09aaf0 con 0x7f42f40b9ca0 2024-09-17T03:14:39.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.965+0000 7f42f9ced640 1 --2- 172.21.15.87:0/521707056 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f42d8078b90 0x7f42d807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:39.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.966+0000 7f42f9ced640 1 --2- 172.21.15.87:0/521707056 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f42d8078b90 0x7f42d807b0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009ac0 tx=0x7f42f0005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:39.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:39.970+0000 7f42e67fc640 1 -- 172.21.15.87:0/521707056 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f42ec062f40 con 0x7f42f40b9ca0 2024-09-17T03:14:40.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.143+0000 7f42fbf51640 1 -- 172.21.15.87:0/521707056 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f42f40cf8c0 con 0x7f42f40b9ca0 2024-09-17T03:14:40.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.147+0000 7f42e67fc640 1 -- 172.21.15.87:0/521707056 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7f42ec062840 con 0x7f42f40b9ca0 2024-09-17T03:14:40.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.149+0000 7f42fbf51640 1 -- 172.21.15.87:0/521707056 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f42d8078b90 msgr2=0x7f42d807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:40.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.149+0000 7f42fbf51640 1 --2- 172.21.15.87:0/521707056 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f42d8078b90 0x7f42d807b0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009ac0 tx=0x7f42f0005f90 comp rx=0 tx=0).stop 2024-09-17T03:14:40.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.149+0000 7f42fbf51640 1 -- 172.21.15.87:0/521707056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42f40b9ca0 msgr2=0x7f42f40cc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:40.148 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.149+0000 7f42fbf51640 1 --2- 172.21.15.87:0/521707056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42f40b9ca0 0x7f42f40cc870 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f42ec00c6c0 tx=0x7f42ec00cb80 comp rx=0 tx=0).stop 2024-09-17T03:14:40.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.149+0000 7f42fbf51640 1 -- 172.21.15.87:0/521707056 shutdown_connections 2024-09-17T03:14:40.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.149+0000 7f42fbf51640 1 --2- 172.21.15.87:0/521707056 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f42d8078b90 0x7f42d807b0a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:40.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.149+0000 7f42fbf51640 1 --2- 172.21.15.87:0/521707056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42f40b9ca0 0x7f42f40cc870 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:40.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.149+0000 7f42fbf51640 1 --2- 172.21.15.87:0/521707056 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f42f40b96f0 0x7f42f40cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:40.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.149+0000 7f42fbf51640 1 --2- 172.21.15.87:0/521707056 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f42f406e020 0x7f42f415dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:40.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.149+0000 7f42fbf51640 1 -- 172.21.15.87:0/521707056 >> 172.21.15.87:0/521707056 conn(0x7f42f409d840 msgr2=0x7f42f4161c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:40.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.149+0000 7f42fbf51640 1 -- 172.21.15.87:0/521707056 shutdown_connections 2024-09-17T03:14:40.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.149+0000 7f42fbf51640 1 -- 172.21.15.87:0/521707056 wait complete. 2024-09-17T03:14:40.160 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T03:14:40.160 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:14:40.203 DEBUG:teuthology.orchestra.run.smithi149:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:14:40.264 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T03:14:40.264 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:14:40.264 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:14:40.280 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:14:40.281 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:14:40.337 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:14:40.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.492+0000 7fd27a9a8640 1 -- 172.21.15.87:0/1069401044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd274072120 msgr2=0x7fd27415efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:40.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.492+0000 7fd27a9a8640 1 --2- 172.21.15.87:0/1069401044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd274072120 0x7fd27415efd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd264009830 tx=0x7fd26402a510 comp rx=0 tx=0).stop 2024-09-17T03:14:40.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.492+0000 7fd27a9a8640 1 -- 172.21.15.87:0/1069401044 shutdown_connections 2024-09-17T03:14:40.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.492+0000 7fd27a9a8640 1 --2- 172.21.15.87:0/1069401044 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd2740b4820 0x7fd27415f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:40.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.492+0000 7fd27a9a8640 1 --2- 172.21.15.87:0/1069401044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd274072120 0x7fd27415efd0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:40.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.492+0000 7fd27a9a8640 1 -- 172.21.15.87:0/1069401044 >> 172.21.15.87:0/1069401044 conn(0x7fd2740d1180 msgr2=0x7fd2740d1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:40.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.492+0000 7fd27a9a8640 1 -- 172.21.15.87:0/1069401044 shutdown_connections 2024-09-17T03:14:40.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.492+0000 7fd27a9a8640 1 -- 172.21.15.87:0/1069401044 wait complete. 2024-09-17T03:14:40.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.494+0000 7fd27a9a8640 1 Processor -- start 2024-09-17T03:14:40.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.494+0000 7fd27a9a8640 1 -- start start 2024-09-17T03:14:40.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.495+0000 7fd27a9a8640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd274072120 0x7fd274151c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:40.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.495+0000 7fd27a9a8640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd274072530 0x7fd274142ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:40.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.495+0000 7fd27a9a8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2740b4820 0x7fd2741e9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:40.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.495+0000 7fd27a9a8640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd274074210 con 0x7fd2740b4820 2024-09-17T03:14:40.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.495+0000 7fd27a9a8640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd274074090 con 0x7fd274072530 2024-09-17T03:14:40.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.495+0000 7fd27a9a8640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd274074390 con 0x7fd274072120 2024-09-17T03:14:40.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.495+0000 7fd278f45640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2740b4820 0x7fd2741e9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:40.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.495+0000 7fd278f45640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2740b4820 0x7fd2741e9190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58240/0 (socket says 172.21.15.87:58240) 2024-09-17T03:14:40.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.495+0000 7fd278f45640 1 -- 172.21.15.87:0/3652797098 learned_addr learned my addr 172.21.15.87:0/3652797098 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:40.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.495+0000 7fd2737fe640 1 --2- 172.21.15.87:0/3652797098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd274072530 0x7fd274142ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:40.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.495+0000 7fd273fff640 1 --2- 172.21.15.87:0/3652797098 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd274072120 0x7fd274151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:40.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.496+0000 7fd278f45640 1 -- 172.21.15.87:0/3652797098 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd274072120 msgr2=0x7fd274151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:40.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.496+0000 7fd278f45640 1 --2- 172.21.15.87:0/3652797098 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd274072120 0x7fd274151c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:40.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.496+0000 7fd278f45640 1 -- 172.21.15.87:0/3652797098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd274072530 msgr2=0x7fd274142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:40.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.496+0000 7fd278f45640 1 --2- 172.21.15.87:0/3652797098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd274072530 0x7fd274142ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:40.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.496+0000 7fd278f45640 1 -- 172.21.15.87:0/3652797098 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2741f8dd0 con 0x7fd2740b4820 2024-09-17T03:14:40.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.496+0000 7fd278f45640 1 --2- 172.21.15.87:0/3652797098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2740b4820 0x7fd2741e9190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00cab0 tx=0x7fd26c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:40.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.496+0000 7fd270ff9640 1 -- 172.21.15.87:0/3652797098 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c007d90 con 0x7fd2740b4820 2024-09-17T03:14:40.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.496+0000 7fd270ff9640 1 -- 172.21.15.87:0/3652797098 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd26c00f560 con 0x7fd2740b4820 2024-09-17T03:14:40.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.496+0000 7fd27a9a8640 1 -- 172.21.15.87:0/3652797098 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2741f90b0 con 0x7fd2740b4820 2024-09-17T03:14:40.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.496+0000 7fd270ff9640 1 -- 172.21.15.87:0/3652797098 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26c00f9b0 con 0x7fd2740b4820 2024-09-17T03:14:40.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.496+0000 7fd27a9a8640 1 -- 172.21.15.87:0/3652797098 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2741f9670 con 0x7fd2740b4820 2024-09-17T03:14:40.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.497+0000 7fd27a9a8640 1 -- 172.21.15.87:0/3652797098 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd274074840 con 0x7fd2740b4820 2024-09-17T03:14:40.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.498+0000 7fd270ff9640 1 -- 172.21.15.87:0/3652797098 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd26c00fb50 con 0x7fd2740b4820 2024-09-17T03:14:40.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.498+0000 7fd270ff9640 1 --2- 172.21.15.87:0/3652797098 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd260078bc0 0x7fd26007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:40.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.498+0000 7fd270ff9640 1 -- 172.21.15.87:0/3652797098 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd26c09b7b0 con 0x7fd2740b4820 2024-09-17T03:14:40.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.499+0000 7fd273fff640 1 --2- 172.21.15.87:0/3652797098 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd260078bc0 0x7fd26007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:40.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.499+0000 7fd273fff640 1 --2- 172.21.15.87:0/3652797098 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd260078bc0 0x7fd26007b0d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd264034000 tx=0x7fd264004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:40.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.503+0000 7fd270ff9640 1 -- 172.21.15.87:0/3652797098 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd26c014070 con 0x7fd2740b4820 2024-09-17T03:14:40.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.692+0000 7fd27a9a8640 1 -- 172.21.15.87:0/3652797098 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd27415fe50 con 0x7fd2740b4820 2024-09-17T03:14:40.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.692+0000 7fd270ff9640 1 -- 172.21.15.87:0/3652797098 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fd26c063500 con 0x7fd2740b4820 2024-09-17T03:14:40.692 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-09-17T03:14:40.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.695+0000 7fd27a9a8640 1 -- 172.21.15.87:0/3652797098 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd260078bc0 msgr2=0x7fd26007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:40.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.695+0000 7fd27a9a8640 1 --2- 172.21.15.87:0/3652797098 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd260078bc0 0x7fd26007b0d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd264034000 tx=0x7fd264004730 comp rx=0 tx=0).stop 2024-09-17T03:14:40.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.695+0000 7fd27a9a8640 1 -- 172.21.15.87:0/3652797098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2740b4820 msgr2=0x7fd2741e9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:40.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.695+0000 7fd27a9a8640 1 --2- 172.21.15.87:0/3652797098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2740b4820 0x7fd2741e9190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00cab0 tx=0x7fd26c00cf70 comp rx=0 tx=0).stop 2024-09-17T03:14:40.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.695+0000 7fd27a9a8640 1 -- 172.21.15.87:0/3652797098 shutdown_connections 2024-09-17T03:14:40.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.695+0000 7fd27a9a8640 1 --2- 172.21.15.87:0/3652797098 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd260078bc0 0x7fd26007b0d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:40.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.695+0000 7fd27a9a8640 1 --2- 172.21.15.87:0/3652797098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2740b4820 0x7fd2741e9190 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:40.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.695+0000 7fd27a9a8640 1 --2- 172.21.15.87:0/3652797098 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd274072530 0x7fd274142ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:40.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.695+0000 7fd27a9a8640 1 --2- 172.21.15.87:0/3652797098 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd274072120 0x7fd274151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:40.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.695+0000 7fd27a9a8640 1 -- 172.21.15.87:0/3652797098 >> 172.21.15.87:0/3652797098 conn(0x7fd2740d1180 msgr2=0x7fd27406d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:40.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.695+0000 7fd27a9a8640 1 -- 172.21.15.87:0/3652797098 shutdown_connections 2024-09-17T03:14:40.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.695+0000 7fd27a9a8640 1 -- 172.21.15.87:0/3652797098 wait complete. 2024-09-17T03:14:40.705 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:14:40.705 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:14:40.721 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:14:40.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.878+0000 7ff1e2dba640 1 -- 172.21.15.87:0/2793703976 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c8003a00 msgr2=0x7ff1c8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:40.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.878+0000 7ff1e2dba640 1 --2- 172.21.15.87:0/2793703976 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c8003a00 0x7ff1c8003e70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400c4a0 tx=0x7ff1c8005830 comp rx=0 tx=0).stop 2024-09-17T03:14:40.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.878+0000 7ff1e1357640 1 -- 172.21.15.87:0/2793703976 reap_dead start 2024-09-17T03:14:40.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.878+0000 7ff1e2dba640 1 -- 172.21.15.87:0/2793703976 shutdown_connections 2024-09-17T03:14:40.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.878+0000 7ff1e2dba640 1 -- 172.21.15.87:0/2793703976 >> 172.21.15.87:0/2793703976 conn(0x7ff1dc0b2d10 msgr2=0x7ff1dc0b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:40.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.879+0000 7ff1e2dba640 1 -- 172.21.15.87:0/2793703976 shutdown_connections 2024-09-17T03:14:40.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.879+0000 7ff1e2dba640 1 -- 172.21.15.87:0/2793703976 wait complete. 2024-09-17T03:14:40.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.880+0000 7ff1e2dba640 1 Processor -- start 2024-09-17T03:14:40.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.880+0000 7ff1e2dba640 1 -- start start 2024-09-17T03:14:40.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.882+0000 7ff1e2dba640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c8003a00 0x7ff1dc142270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:40.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.882+0000 7ff1e2dba640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1dc158860 0x7ff1dc06cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:40.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.882+0000 7ff1e2dba640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1dc06d320 0x7ff1dc144a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:40.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.882+0000 7ff1e2dba640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc15bd10 con 0x7ff1c8003a00 2024-09-17T03:14:40.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.882+0000 7ff1e2dba640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff1dc15bb90 con 0x7ff1dc158860 2024-09-17T03:14:40.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.882+0000 7ff1e2dba640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff1dc15be90 con 0x7ff1dc06d320 2024-09-17T03:14:40.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.882+0000 7ff1e0b56640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c8003a00 0x7ff1dc142270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:40.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.882+0000 7ff1e0b56640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c8003a00 0x7ff1dc142270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58258/0 (socket says 172.21.15.87:58258) 2024-09-17T03:14:40.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.882+0000 7ff1e0b56640 1 -- 172.21.15.87:0/2844451325 learned_addr learned my addr 172.21.15.87:0/2844451325 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:40.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.882+0000 7ff1dbfff640 1 --2- 172.21.15.87:0/2844451325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1dc158860 0x7ff1dc06cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:40.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.882+0000 7ff1e1357640 1 --2- 172.21.15.87:0/2844451325 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1dc06d320 0x7ff1dc144a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:40.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.882+0000 7ff1e0b56640 1 -- 172.21.15.87:0/2844451325 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1dc06d320 msgr2=0x7ff1dc144a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:40.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.882+0000 7ff1e0b56640 1 --2- 172.21.15.87:0/2844451325 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1dc06d320 0x7ff1dc144a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:40.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.882+0000 7ff1e0b56640 1 -- 172.21.15.87:0/2844451325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1dc158860 msgr2=0x7ff1dc06cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:40.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.882+0000 7ff1e0b56640 1 --2- 172.21.15.87:0/2844451325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1dc158860 0x7ff1dc06cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:40.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.882+0000 7ff1e0b56640 1 -- 172.21.15.87:0/2844451325 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1dc06d730 con 0x7ff1c8003a00 2024-09-17T03:14:40.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.882+0000 7ff1e0b56640 1 --2- 172.21.15.87:0/2844451325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c8003a00 0x7ff1dc142270 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400c5c0 tx=0x7ff1d4006270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:40.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.882+0000 7ff1d97fa640 1 -- 172.21.15.87:0/2844451325 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4038030 con 0x7ff1c8003a00 2024-09-17T03:14:40.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.883+0000 7ff1d97fa640 1 -- 172.21.15.87:0/2844451325 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1d4001180 con 0x7ff1c8003a00 2024-09-17T03:14:40.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.883+0000 7ff1e2dba640 1 -- 172.21.15.87:0/2844451325 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1dc155150 con 0x7ff1c8003a00 2024-09-17T03:14:40.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.883+0000 7ff1d97fa640 1 -- 172.21.15.87:0/2844451325 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d402b8c0 con 0x7ff1c8003a00 2024-09-17T03:14:40.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.883+0000 7ff1e2dba640 1 -- 172.21.15.87:0/2844451325 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1dc142e90 con 0x7ff1c8003a00 2024-09-17T03:14:40.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.884+0000 7ff1e2dba640 1 -- 172.21.15.87:0/2844451325 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1dc15f180 con 0x7ff1c8003a00 2024-09-17T03:14:40.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.884+0000 7ff1d97fa640 1 -- 172.21.15.87:0/2844451325 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff1d403e030 con 0x7ff1c8003a00 2024-09-17T03:14:40.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.885+0000 7ff1d97fa640 1 --2- 172.21.15.87:0/2844451325 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff1bc078bc0 0x7ff1bc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:40.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.885+0000 7ff1d97fa640 1 -- 172.21.15.87:0/2844451325 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff1d40ba3e0 con 0x7ff1c8003a00 2024-09-17T03:14:40.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.885+0000 7ff1dbfff640 1 --2- 172.21.15.87:0/2844451325 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff1bc078bc0 0x7ff1bc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:40.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.886+0000 7ff1dbfff640 1 --2- 172.21.15.87:0/2844451325 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff1bc078bc0 0x7ff1bc07b0d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0b33d0 tx=0x7ff1cc009500 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:40.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:40.889+0000 7ff1d97fa640 1 -- 172.21.15.87:0/2844451325 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff1dc06d730 con 0x7ff1c8003a00 2024-09-17T03:14:41.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.075+0000 7ff1e2dba640 1 -- 172.21.15.87:0/2844451325 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff1dc0b3400 con 0x7ff1c8003a00 2024-09-17T03:14:41.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.075+0000 7ff1d97fa640 1 -- 172.21.15.87:0/2844451325 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7ff1dc15fad0 con 0x7ff1c8003a00 2024-09-17T03:14:41.074 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-09-17T03:14:41.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.077+0000 7ff1e2dba640 1 -- 172.21.15.87:0/2844451325 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff1bc078bc0 msgr2=0x7ff1bc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:41.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.077+0000 7ff1e2dba640 1 --2- 172.21.15.87:0/2844451325 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff1bc078bc0 0x7ff1bc07b0d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0b33d0 tx=0x7ff1cc009500 comp rx=0 tx=0).stop 2024-09-17T03:14:41.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.077+0000 7ff1e2dba640 1 -- 172.21.15.87:0/2844451325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c8003a00 msgr2=0x7ff1dc142270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:41.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.077+0000 7ff1e2dba640 1 --2- 172.21.15.87:0/2844451325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c8003a00 0x7ff1dc142270 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff1d400c5c0 tx=0x7ff1d4006270 comp rx=0 tx=0).stop 2024-09-17T03:14:41.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.077+0000 7ff1e2dba640 1 -- 172.21.15.87:0/2844451325 shutdown_connections 2024-09-17T03:14:41.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.077+0000 7ff1e2dba640 1 --2- 172.21.15.87:0/2844451325 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff1bc078bc0 0x7ff1bc07b0d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:41.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.077+0000 7ff1e2dba640 1 --2- 172.21.15.87:0/2844451325 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff1dc06d320 0x7ff1dc144a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:41.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.077+0000 7ff1e2dba640 1 --2- 172.21.15.87:0/2844451325 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff1dc158860 0x7ff1dc06cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:41.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.077+0000 7ff1e2dba640 1 --2- 172.21.15.87:0/2844451325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1c8003a00 0x7ff1dc142270 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:41.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.077+0000 7ff1e2dba640 1 -- 172.21.15.87:0/2844451325 >> 172.21.15.87:0/2844451325 conn(0x7ff1dc0b2d10 msgr2=0x7ff1dc151740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:41.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.077+0000 7ff1e2dba640 1 -- 172.21.15.87:0/2844451325 shutdown_connections 2024-09-17T03:14:41.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.077+0000 7ff1e2dba640 1 -- 172.21.15.87:0/2844451325 wait complete. 2024-09-17T03:14:41.087 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi087.front.sepia.ceph.com... 2024-09-17T03:14:41.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:41.087 DEBUG:teuthology.orchestra.run.smithi087:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T03:14:41.112 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:14:41.113 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-09-17T03:14:41.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:41.113 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T03:14:41.173 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:14:41.173 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:14:41.193 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:14:41.193 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:14:41.253 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:14:41.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.368+0000 7f47dfc7e640 1 -- 172.21.15.87:0/1340035074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47d80b76d0 msgr2=0x7f47d80cee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:41.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.368+0000 7f47dfc7e640 1 --2- 172.21.15.87:0/1340035074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47d80b76d0 0x7f47d80cee00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f47d8073770 tx=0x7f47d402a530 comp rx=0 tx=0).stop 2024-09-17T03:14:41.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.368+0000 7f47dfc7e640 1 -- 172.21.15.87:0/1340035074 shutdown_connections 2024-09-17T03:14:41.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.368+0000 7f47dfc7e640 1 --2- 172.21.15.87:0/1340035074 >> v2:172.21.15.149:3300/0 conn(0x7f47d80cf340 0x7f47d80d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:41.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.368+0000 7f47dfc7e640 1 --2- 172.21.15.87:0/1340035074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47d80b76d0 0x7f47d80cee00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:41.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.368+0000 7f47dfc7e640 1 -- 172.21.15.87:0/1340035074 >> 172.21.15.87:0/1340035074 conn(0x7f47d80d2230 msgr2=0x7f47d80d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:41.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.368+0000 7f47dfc7e640 1 -- 172.21.15.87:0/1340035074 shutdown_connections 2024-09-17T03:14:41.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.368+0000 7f47dfc7e640 1 -- 172.21.15.87:0/1340035074 wait complete. 2024-09-17T03:14:41.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.370+0000 7f47dfc7e640 1 Processor -- start 2024-09-17T03:14:41.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.370+0000 7f47dfc7e640 1 -- start start 2024-09-17T03:14:41.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.371+0000 7f47dfc7e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47d80b76d0 0x7f47d80b6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:41.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.371+0000 7f47dfc7e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47d80cf340 0x7f47d80b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:41.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.371+0000 7f47dfc7e640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f47d80d1080 0x7f47d81e5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:41.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.371+0000 7f47dfc7e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f47d80cd810 con 0x7f47d80cf340 2024-09-17T03:14:41.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.371+0000 7f47dfc7e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f47d80cd690 con 0x7f47d80b76d0 2024-09-17T03:14:41.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.371+0000 7f47dfc7e640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f47d80b19d0 con 0x7f47d80d1080 2024-09-17T03:14:41.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.372+0000 7f47dd219640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47d80cf340 0x7f47d80b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:41.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.372+0000 7f47dd219640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47d80cf340 0x7f47d80b0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58280/0 (socket says 172.21.15.87:58280) 2024-09-17T03:14:41.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.372+0000 7f47dd219640 1 -- 172.21.15.87:0/1236089941 learned_addr learned my addr 172.21.15.87:0/1236089941 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:41.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.372+0000 7f47dda1a640 1 --2- 172.21.15.87:0/1236089941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47d80b76d0 0x7f47d80b6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:41.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.372+0000 7f47de21b640 1 --2- 172.21.15.87:0/1236089941 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f47d80d1080 0x7f47d81e5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:41.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.372+0000 7f47dd219640 1 -- 172.21.15.87:0/1236089941 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f47d80d1080 msgr2=0x7f47d81e5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:41.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.372+0000 7f47dd219640 1 --2- 172.21.15.87:0/1236089941 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f47d80d1080 0x7f47d81e5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:41.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.372+0000 7f47dd219640 1 -- 172.21.15.87:0/1236089941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47d80b76d0 msgr2=0x7f47d80b6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:41.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.372+0000 7f47dd219640 1 --2- 172.21.15.87:0/1236089941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47d80b76d0 0x7f47d80b6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:41.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.372+0000 7f47dd219640 1 -- 172.21.15.87:0/1236089941 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d81f6db0 con 0x7f47d80cf340 2024-09-17T03:14:41.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.372+0000 7f47dd219640 1 --2- 172.21.15.87:0/1236089941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47d80cf340 0x7f47d80b0200 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f47c800c970 tx=0x7f47c800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:41.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.372+0000 7f47ce7fc640 1 -- 172.21.15.87:0/1236089941 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c8007bb0 con 0x7f47d80cf340 2024-09-17T03:14:41.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.372+0000 7f47ce7fc640 1 -- 172.21.15.87:0/1236089941 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47c8007e40 con 0x7f47d80cf340 2024-09-17T03:14:41.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.372+0000 7f47dfc7e640 1 -- 172.21.15.87:0/1236089941 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d81f7090 con 0x7f47d80cf340 2024-09-17T03:14:41.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.372+0000 7f47ce7fc640 1 -- 172.21.15.87:0/1236089941 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c8005280 con 0x7f47d80cf340 2024-09-17T03:14:41.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.372+0000 7f47dfc7e640 1 -- 172.21.15.87:0/1236089941 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d81f7650 con 0x7f47d80cf340 2024-09-17T03:14:41.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.373+0000 7f47dfc7e640 1 -- 172.21.15.87:0/1236089941 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47d80cee00 con 0x7f47d80cf340 2024-09-17T03:14:41.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.374+0000 7f47ce7fc640 1 -- 172.21.15.87:0/1236089941 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f47c8005420 con 0x7f47d80cf340 2024-09-17T03:14:41.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.375+0000 7f47ce7fc640 1 --2- 172.21.15.87:0/1236089941 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f47bc078bc0 0x7f47bc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:41.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.375+0000 7f47ce7fc640 1 -- 172.21.15.87:0/1236089941 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f47c809b190 con 0x7f47d80cf340 2024-09-17T03:14:41.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.375+0000 7f47dda1a640 1 --2- 172.21.15.87:0/1236089941 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f47bc078bc0 0x7f47bc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:41.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.376+0000 7f47dda1a640 1 --2- 172.21.15.87:0/1236089941 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f47bc078bc0 0x7f47bc07b0d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f47d4034000 tx=0x7f47d4004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:41.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.379+0000 7f47ce7fc640 1 -- 172.21.15.87:0/1236089941 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f47c8014030 con 0x7f47d80cf340 2024-09-17T03:14:41.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.571+0000 7f47dfc7e640 1 -- 172.21.15.87:0/1236089941 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f47d80d2a90 con 0x7f47d80cf340 2024-09-17T03:14:41.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.572+0000 7f47ce7fc640 1 -- 172.21.15.87:0/1236089941 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f47c8063040 con 0x7f47d80cf340 2024-09-17T03:14:41.571 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-09-17T03:14:41.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.574+0000 7f47dfc7e640 1 -- 172.21.15.87:0/1236089941 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f47bc078bc0 msgr2=0x7f47bc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:41.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.574+0000 7f47dfc7e640 1 --2- 172.21.15.87:0/1236089941 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f47bc078bc0 0x7f47bc07b0d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f47d4034000 tx=0x7f47d4004730 comp rx=0 tx=0).stop 2024-09-17T03:14:41.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.574+0000 7f47dfc7e640 1 -- 172.21.15.87:0/1236089941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47d80cf340 msgr2=0x7f47d80b0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:41.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.574+0000 7f47dfc7e640 1 --2- 172.21.15.87:0/1236089941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47d80cf340 0x7f47d80b0200 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f47c800c970 tx=0x7f47c800ce30 comp rx=0 tx=0).stop 2024-09-17T03:14:41.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.574+0000 7f47dfc7e640 1 -- 172.21.15.87:0/1236089941 shutdown_connections 2024-09-17T03:14:41.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.574+0000 7f47dfc7e640 1 --2- 172.21.15.87:0/1236089941 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f47bc078bc0 0x7f47bc07b0d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:41.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.574+0000 7f47dfc7e640 1 --2- 172.21.15.87:0/1236089941 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f47d80d1080 0x7f47d81e5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:41.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.574+0000 7f47dfc7e640 1 --2- 172.21.15.87:0/1236089941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47d80cf340 0x7f47d80b0200 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:41.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.574+0000 7f47dfc7e640 1 --2- 172.21.15.87:0/1236089941 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f47d80b76d0 0x7f47d80b6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:41.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.574+0000 7f47dfc7e640 1 -- 172.21.15.87:0/1236089941 >> 172.21.15.87:0/1236089941 conn(0x7f47d80d2230 msgr2=0x7f47d80ccf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:41.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.574+0000 7f47dfc7e640 1 -- 172.21.15.87:0/1236089941 shutdown_connections 2024-09-17T03:14:41.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.574+0000 7f47dfc7e640 1 -- 172.21.15.87:0/1236089941 wait complete. 2024-09-17T03:14:41.584 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:14:41.584 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:14:41.603 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:14:41.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.718+0000 7f70289ad640 1 -- 172.21.15.87:0/3706025153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f702415c5f0 msgr2=0x7f702415f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:41.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.718+0000 7f70289ad640 1 --2- 172.21.15.87:0/3706025153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f702415c5f0 0x7f702415f890 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7014009830 tx=0x7f701402a510 comp rx=0 tx=0).stop 2024-09-17T03:14:41.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.718+0000 7f70289ad640 1 -- 172.21.15.87:0/3706025153 shutdown_connections 2024-09-17T03:14:41.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.718+0000 7f70289ad640 1 --2- 172.21.15.87:0/3706025153 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70240cf020 0x7f70240b5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:41.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.718+0000 7f70289ad640 1 --2- 172.21.15.87:0/3706025153 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f702415c5f0 0x7f702415f890 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:41.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.718+0000 7f70289ad640 1 -- 172.21.15.87:0/3706025153 >> 172.21.15.87:0/3706025153 conn(0x7f7024069900 msgr2=0x7f70240d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:41.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.718+0000 7f70289ad640 1 -- 172.21.15.87:0/3706025153 shutdown_connections 2024-09-17T03:14:41.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.718+0000 7f70289ad640 1 -- 172.21.15.87:0/3706025153 wait complete. 2024-09-17T03:14:41.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.720+0000 7f70289ad640 1 Processor -- start 2024-09-17T03:14:41.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.720+0000 7f70289ad640 1 -- start start 2024-09-17T03:14:41.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.721+0000 7f70289ad640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70240cf020 0x7f70241f5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:41.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.721+0000 7f70289ad640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70240cf430 0x7f70240c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:41.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.721+0000 7f70289ad640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f702415c5f0 0x7f702406e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:41.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.721+0000 7f70289ad640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f70240b6770 con 0x7f702415c5f0 2024-09-17T03:14:41.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.721+0000 7f70289ad640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f70240b65f0 con 0x7f70240cf020 2024-09-17T03:14:41.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.721+0000 7f70289ad640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f70241627b0 con 0x7f70240cf430 2024-09-17T03:14:41.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.722+0000 7f7022d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f702415c5f0 0x7f702406e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:41.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.722+0000 7f7022d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f702415c5f0 0x7f702406e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58294/0 (socket says 172.21.15.87:58294) 2024-09-17T03:14:41.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.722+0000 7f7022d9d640 1 -- 172.21.15.87:0/3234593688 learned_addr learned my addr 172.21.15.87:0/3234593688 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:41.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.722+0000 7f702259c640 1 --2- 172.21.15.87:0/3234593688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70240cf020 0x7f70241f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:41.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.722+0000 7f7022d9d640 1 -- 172.21.15.87:0/3234593688 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70240cf430 msgr2=0x7f70240c7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:41.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.722+0000 7f7022d9d640 1 --2- 172.21.15.87:0/3234593688 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70240cf430 0x7f70240c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:41.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.722+0000 7f7022d9d640 1 -- 172.21.15.87:0/3234593688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70240cf020 msgr2=0x7f70241f5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:41.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.722+0000 7f7022d9d640 1 --2- 172.21.15.87:0/3234593688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70240cf020 0x7f70241f5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:41.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.722+0000 7f7022d9d640 1 -- 172.21.15.87:0/3234593688 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70241f62b0 con 0x7f702415c5f0 2024-09-17T03:14:41.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.722+0000 7f7022d9d640 1 --2- 172.21.15.87:0/3234593688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f702415c5f0 0x7f702406e060 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f701c00cae0 tx=0x7f701c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:41.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.723+0000 7f7012ffd640 1 -- 172.21.15.87:0/3234593688 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c016020 con 0x7f702415c5f0 2024-09-17T03:14:41.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.723+0000 7f7012ffd640 1 -- 172.21.15.87:0/3234593688 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f701c00ad30 con 0x7f702415c5f0 2024-09-17T03:14:41.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.723+0000 7f7012ffd640 1 -- 172.21.15.87:0/3234593688 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c012680 con 0x7f702415c5f0 2024-09-17T03:14:41.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.723+0000 7f70289ad640 1 -- 172.21.15.87:0/3234593688 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70241f6590 con 0x7f702415c5f0 2024-09-17T03:14:41.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.723+0000 7f70289ad640 1 -- 172.21.15.87:0/3234593688 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70240cc550 con 0x7f702415c5f0 2024-09-17T03:14:41.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.724+0000 7f70289ad640 1 -- 172.21.15.87:0/3234593688 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70240cc990 con 0x7f702415c5f0 2024-09-17T03:14:41.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.725+0000 7f7012ffd640 1 -- 172.21.15.87:0/3234593688 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f701c019020 con 0x7f702415c5f0 2024-09-17T03:14:41.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.725+0000 7f7012ffd640 1 --2- 172.21.15.87:0/3234593688 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7004078bc0 0x7f700407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:41.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.725+0000 7f7012ffd640 1 -- 172.21.15.87:0/3234593688 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f701c09af40 con 0x7f702415c5f0 2024-09-17T03:14:41.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.725+0000 7f702259c640 1 --2- 172.21.15.87:0/3234593688 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7004078bc0 0x7f700407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:41.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.726+0000 7f702259c640 1 --2- 172.21.15.87:0/3234593688 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7004078bc0 0x7f700407b0d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7014034000 tx=0x7f7014004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:41.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.730+0000 7f7012ffd640 1 -- 172.21.15.87:0/3234593688 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f701c020030 con 0x7f702415c5f0 2024-09-17T03:14:41.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.915+0000 7f70289ad640 1 -- 172.21.15.87:0/3234593688 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f70241f6040 con 0x7f702415c5f0 2024-09-17T03:14:41.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.916+0000 7f7012ffd640 1 -- 172.21.15.87:0/3234593688 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f701c063410 con 0x7f702415c5f0 2024-09-17T03:14:41.917 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-09-17T03:14:41.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.918+0000 7f70289ad640 1 -- 172.21.15.87:0/3234593688 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7004078bc0 msgr2=0x7f700407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:41.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.918+0000 7f70289ad640 1 --2- 172.21.15.87:0/3234593688 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7004078bc0 0x7f700407b0d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7014034000 tx=0x7f7014004730 comp rx=0 tx=0).stop 2024-09-17T03:14:41.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.918+0000 7f70289ad640 1 -- 172.21.15.87:0/3234593688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f702415c5f0 msgr2=0x7f702406e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:41.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.918+0000 7f70289ad640 1 --2- 172.21.15.87:0/3234593688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f702415c5f0 0x7f702406e060 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f701c00cae0 tx=0x7f701c00a060 comp rx=0 tx=0).stop 2024-09-17T03:14:41.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.918+0000 7f70289ad640 1 -- 172.21.15.87:0/3234593688 shutdown_connections 2024-09-17T03:14:41.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.918+0000 7f70289ad640 1 --2- 172.21.15.87:0/3234593688 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7004078bc0 0x7f700407b0d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:41.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.918+0000 7f70289ad640 1 --2- 172.21.15.87:0/3234593688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f702415c5f0 0x7f702406e060 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:41.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.918+0000 7f70289ad640 1 --2- 172.21.15.87:0/3234593688 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f70240cf430 0x7f70240c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:41.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.918+0000 7f70289ad640 1 --2- 172.21.15.87:0/3234593688 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f70240cf020 0x7f70241f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:41.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.918+0000 7f70289ad640 1 -- 172.21.15.87:0/3234593688 >> 172.21.15.87:0/3234593688 conn(0x7f7024069900 msgr2=0x7f7024160590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:41.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.918+0000 7f70289ad640 1 -- 172.21.15.87:0/3234593688 shutdown_connections 2024-09-17T03:14:41.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:41.918+0000 7f70289ad640 1 -- 172.21.15.87:0/3234593688 wait complete. 2024-09-17T03:14:41.929 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi149.front.sepia.ceph.com... 2024-09-17T03:14:41.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:41.929 DEBUG:teuthology.orchestra.run.smithi149:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-17T03:14:41.960 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:14:41.960 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi149.front.sepia.ceph.com 2024-09-17T03:14:41.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:41.960 DEBUG:teuthology.orchestra.run.smithi149:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-17T03:14:42.033 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:14:42.033 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:14:42.049 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:14:42.049 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:14:42.105 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:14:42.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.260+0000 7f9fe0463640 1 -- 172.21.15.87:0/3067053374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fd806d320 msgr2=0x7f9fd80cd950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:42.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.260+0000 7f9fe0463640 1 --2- 172.21.15.87:0/3067053374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fd806d320 0x7f9fd80cd950 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0005fd0 tx=0x7f9fd002aa00 comp rx=0 tx=0).stop 2024-09-17T03:14:42.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.261+0000 7f9fe0463640 1 -- 172.21.15.87:0/3067053374 shutdown_connections 2024-09-17T03:14:42.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.261+0000 7f9fe0463640 1 --2- 172.21.15.87:0/3067053374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fd806d320 0x7f9fd80cd950 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:42.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.261+0000 7f9fe0463640 1 --2- 172.21.15.87:0/3067053374 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9fd8160be0 0x7f9fd806cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:42.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.261+0000 7f9fe0463640 1 -- 172.21.15.87:0/3067053374 >> 172.21.15.87:0/3067053374 conn(0x7f9fd80d1180 msgr2=0x7f9fd80d1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:42.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.261+0000 7f9fe0463640 1 -- 172.21.15.87:0/3067053374 shutdown_connections 2024-09-17T03:14:42.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.261+0000 7f9fe0463640 1 -- 172.21.15.87:0/3067053374 wait complete. 2024-09-17T03:14:42.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.262+0000 7f9fe0463640 1 Processor -- start 2024-09-17T03:14:42.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.263+0000 7f9fe0463640 1 -- start start 2024-09-17T03:14:42.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.264+0000 7f9fe0463640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9fd806d320 0x7f9fd81f8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:42.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.264+0000 7f9fe0463640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9fd80d0c90 0x7f9fd81f5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:42.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.264+0000 7f9fe0463640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fd8160be0 0x7f9fd8142be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:42.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.264+0000 7f9fe0463640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd80ce010 con 0x7f9fd8160be0 2024-09-17T03:14:42.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.264+0000 7f9fe0463640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd80cde90 con 0x7f9fd806d320 2024-09-17T03:14:42.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.264+0000 7f9fe0463640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9fd80ce190 con 0x7f9fd80d0c90 2024-09-17T03:14:42.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.264+0000 7f9fdea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fd8160be0 0x7f9fd8142be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:42.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.264+0000 7f9fdea00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fd8160be0 0x7f9fd8142be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58320/0 (socket says 172.21.15.87:58320) 2024-09-17T03:14:42.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.264+0000 7f9fdea00640 1 -- 172.21.15.87:0/728905564 learned_addr learned my addr 172.21.15.87:0/728905564 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:42.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.264+0000 7f9fde1ff640 1 --2- 172.21.15.87:0/728905564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9fd806d320 0x7f9fd81f8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:42.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.264+0000 7f9fdd9fe640 1 --2- 172.21.15.87:0/728905564 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9fd80d0c90 0x7f9fd81f5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:42.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.264+0000 7f9fdea00640 1 -- 172.21.15.87:0/728905564 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9fd80d0c90 msgr2=0x7f9fd81f5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:42.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.264+0000 7f9fdea00640 1 --2- 172.21.15.87:0/728905564 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9fd80d0c90 0x7f9fd81f5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:42.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.264+0000 7f9fdea00640 1 -- 172.21.15.87:0/728905564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9fd806d320 msgr2=0x7f9fd81f8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:42.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.264+0000 7f9fdea00640 1 --2- 172.21.15.87:0/728905564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9fd806d320 0x7f9fd81f8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:42.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.264+0000 7f9fdea00640 1 -- 172.21.15.87:0/728905564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd81f97f0 con 0x7f9fd8160be0 2024-09-17T03:14:42.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.264+0000 7f9fdea00640 1 --2- 172.21.15.87:0/728905564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fd8160be0 0x7f9fd8142be0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0006010 tx=0x7f9fd0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:42.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.265+0000 7f9fceffd640 1 -- 172.21.15.87:0/728905564 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0046070 con 0x7f9fd8160be0 2024-09-17T03:14:42.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.265+0000 7f9fceffd640 1 -- 172.21.15.87:0/728905564 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fd0004540 con 0x7f9fd8160be0 2024-09-17T03:14:42.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.265+0000 7f9fceffd640 1 -- 172.21.15.87:0/728905564 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0033070 con 0x7f9fd8160be0 2024-09-17T03:14:42.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.265+0000 7f9fe0463640 1 -- 172.21.15.87:0/728905564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd81f9ad0 con 0x7f9fd8160be0 2024-09-17T03:14:42.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.265+0000 7f9fe0463640 1 -- 172.21.15.87:0/728905564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd81f9db0 con 0x7f9fd8160be0 2024-09-17T03:14:42.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.266+0000 7f9fe0463640 1 -- 172.21.15.87:0/728905564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fd81f9560 con 0x7f9fd8160be0 2024-09-17T03:14:42.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.266+0000 7f9fceffd640 1 -- 172.21.15.87:0/728905564 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9fd0049070 con 0x7f9fd8160be0 2024-09-17T03:14:42.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.267+0000 7f9fceffd640 1 --2- 172.21.15.87:0/728905564 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9fbc078b90 0x7f9fbc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:42.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.267+0000 7f9fceffd640 1 -- 172.21.15.87:0/728905564 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f9fd00bbdb0 con 0x7f9fd8160be0 2024-09-17T03:14:42.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.267+0000 7f9fde1ff640 1 --2- 172.21.15.87:0/728905564 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9fbc078b90 0x7f9fbc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:42.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.268+0000 7f9fde1ff640 1 --2- 172.21.15.87:0/728905564 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9fbc078b90 0x7f9fbc07b0a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4006fd0 tx=0x7f9fd4009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:42.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.272+0000 7f9fceffd640 1 -- 172.21.15.87:0/728905564 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9fd003e070 con 0x7f9fd8160be0 2024-09-17T03:14:42.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.457+0000 7f9fe0463640 1 -- 172.21.15.87:0/728905564 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9fd806dac0 con 0x7f9fd8160be0 2024-09-17T03:14:42.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.457+0000 7f9fceffd640 1 -- 172.21.15.87:0/728905564 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f9fd806d730 con 0x7f9fd8160be0 2024-09-17T03:14:42.456 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-09-17T03:14:42.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.459+0000 7f9fe0463640 1 -- 172.21.15.87:0/728905564 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9fbc078b90 msgr2=0x7f9fbc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:42.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.459+0000 7f9fe0463640 1 --2- 172.21.15.87:0/728905564 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9fbc078b90 0x7f9fbc07b0a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4006fd0 tx=0x7f9fd4009060 comp rx=0 tx=0).stop 2024-09-17T03:14:42.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.459+0000 7f9fe0463640 1 -- 172.21.15.87:0/728905564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fd8160be0 msgr2=0x7f9fd8142be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:42.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.459+0000 7f9fe0463640 1 --2- 172.21.15.87:0/728905564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fd8160be0 0x7f9fd8142be0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0006010 tx=0x7f9fd0003f10 comp rx=0 tx=0).stop 2024-09-17T03:14:42.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.459+0000 7f9fe0463640 1 -- 172.21.15.87:0/728905564 shutdown_connections 2024-09-17T03:14:42.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.459+0000 7f9fe0463640 1 --2- 172.21.15.87:0/728905564 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9fbc078b90 0x7f9fbc07b0a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:42.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.459+0000 7f9fe0463640 1 --2- 172.21.15.87:0/728905564 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fd8160be0 0x7f9fd8142be0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:42.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.459+0000 7f9fe0463640 1 --2- 172.21.15.87:0/728905564 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9fd80d0c90 0x7f9fd81f5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:42.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.459+0000 7f9fe0463640 1 --2- 172.21.15.87:0/728905564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9fd806d320 0x7f9fd81f8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:42.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.459+0000 7f9fe0463640 1 -- 172.21.15.87:0/728905564 >> 172.21.15.87:0/728905564 conn(0x7f9fd80d1180 msgr2=0x7f9fd80b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:42.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.459+0000 7f9fe0463640 1 -- 172.21.15.87:0/728905564 shutdown_connections 2024-09-17T03:14:42.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.459+0000 7f9fe0463640 1 -- 172.21.15.87:0/728905564 wait complete. 2024-09-17T03:14:42.469 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:14:42.470 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:14:42.486 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:14:42.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.642+0000 7fe917656640 1 -- 172.21.15.87:0/2298216072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8fc004d40 msgr2=0x7fe8fc0051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:42.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.642+0000 7fe917656640 1 --2- 172.21.15.87:0/2298216072 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8fc004d40 0x7fe8fc0051f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe9100cfa50 tx=0x7fe8fc005830 comp rx=0 tx=0).stop 2024-09-17T03:14:42.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.642+0000 7fe915bf3640 1 -- 172.21.15.87:0/2298216072 reap_dead start 2024-09-17T03:14:42.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.642+0000 7fe917656640 1 -- 172.21.15.87:0/2298216072 shutdown_connections 2024-09-17T03:14:42.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.642+0000 7fe917656640 1 -- 172.21.15.87:0/2298216072 >> 172.21.15.87:0/2298216072 conn(0x7fe910069360 msgr2=0x7fe9100ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:42.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.643+0000 7fe917656640 1 -- 172.21.15.87:0/2298216072 shutdown_connections 2024-09-17T03:14:42.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.643+0000 7fe917656640 1 -- 172.21.15.87:0/2298216072 wait complete. 2024-09-17T03:14:42.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.644+0000 7fe917656640 1 Processor -- start 2024-09-17T03:14:42.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.644+0000 7fe917656640 1 -- start start 2024-09-17T03:14:42.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.646+0000 7fe917656640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8fc004d40 0x7fe9100b24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:42.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.646+0000 7fe917656640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe9100b2a30 0x7fe9101f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:42.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.646+0000 7fe917656640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9101f7390 0x7fe9101f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:42.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.646+0000 7fe917656640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe910160da0 con 0x7fe8fc004d40 2024-09-17T03:14:42.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.646+0000 7fe917656640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe910160c20 con 0x7fe9101f7390 2024-09-17T03:14:42.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.646+0000 7fe917656640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe910160f20 con 0x7fe9100b2a30 2024-09-17T03:14:42.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.646+0000 7fe9153f2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8fc004d40 0x7fe9100b24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:42.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.646+0000 7fe9153f2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8fc004d40 0x7fe9100b24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58340/0 (socket says 172.21.15.87:58340) 2024-09-17T03:14:42.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.646+0000 7fe9153f2640 1 -- 172.21.15.87:0/422937609 learned_addr learned my addr 172.21.15.87:0/422937609 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:42.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.646+0000 7fe914bf1640 1 --2- 172.21.15.87:0/422937609 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe9100b2a30 0x7fe9101f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:42.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.646+0000 7fe915bf3640 1 --2- 172.21.15.87:0/422937609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9101f7390 0x7fe9101f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:42.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.646+0000 7fe9153f2640 1 -- 172.21.15.87:0/422937609 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe9100b2a30 msgr2=0x7fe9101f6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:42.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.646+0000 7fe9153f2640 1 --2- 172.21.15.87:0/422937609 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe9100b2a30 0x7fe9101f6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:42.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.646+0000 7fe9153f2640 1 -- 172.21.15.87:0/422937609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9101f7390 msgr2=0x7fe9101f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:42.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.646+0000 7fe9153f2640 1 --2- 172.21.15.87:0/422937609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9101f7390 0x7fe9101f77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:42.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.646+0000 7fe9153f2640 1 -- 172.21.15.87:0/422937609 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91015d030 con 0x7fe8fc004d40 2024-09-17T03:14:42.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.646+0000 7fe9153f2640 1 --2- 172.21.15.87:0/422937609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8fc004d40 0x7fe9100b24f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe90c011a50 tx=0x7fe90c011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:42.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.646+0000 7fe905ffb640 1 -- 172.21.15.87:0/422937609 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c01f070 con 0x7fe8fc004d40 2024-09-17T03:14:42.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.646+0000 7fe905ffb640 1 -- 172.21.15.87:0/422937609 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe90c00fdc0 con 0x7fe8fc004d40 2024-09-17T03:14:42.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.646+0000 7fe917656640 1 -- 172.21.15.87:0/422937609 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9100b2e40 con 0x7fe8fc004d40 2024-09-17T03:14:42.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.646+0000 7fe905ffb640 1 -- 172.21.15.87:0/422937609 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c0126b0 con 0x7fe8fc004d40 2024-09-17T03:14:42.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.647+0000 7fe917656640 1 -- 172.21.15.87:0/422937609 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe91015d2d0 con 0x7fe8fc004d40 2024-09-17T03:14:42.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.647+0000 7fe917656640 1 -- 172.21.15.87:0/422937609 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9100c8c20 con 0x7fe8fc004d40 2024-09-17T03:14:42.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.648+0000 7fe905ffb640 1 -- 172.21.15.87:0/422937609 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe90c022070 con 0x7fe8fc004d40 2024-09-17T03:14:42.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.649+0000 7fe905ffb640 1 --2- 172.21.15.87:0/422937609 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe8f4078b90 0x7fe8f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:42.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.649+0000 7fe905ffb640 1 -- 172.21.15.87:0/422937609 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe90c09b740 con 0x7fe8fc004d40 2024-09-17T03:14:42.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.649+0000 7fe914bf1640 1 --2- 172.21.15.87:0/422937609 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe8f4078b90 0x7fe8f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:42.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.650+0000 7fe914bf1640 1 --2- 172.21.15.87:0/422937609 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe8f4078b90 0x7fe8f407b0a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe900000f60 tx=0x7fe900000ef0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:42.652 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.654+0000 7fe905ffb640 1 -- 172.21.15.87:0/422937609 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe9100c8c20 con 0x7fe8fc004d40 2024-09-17T03:14:42.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.839+0000 7fe917656640 1 -- 172.21.15.87:0/422937609 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe9100ce220 con 0x7fe8fc004d40 2024-09-17T03:14:42.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.839+0000 7fe905ffb640 1 -- 172.21.15.87:0/422937609 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fe9100ce220 con 0x7fe8fc004d40 2024-09-17T03:14:42.838 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-09-17T03:14:42.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.841+0000 7fe917656640 1 -- 172.21.15.87:0/422937609 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe8f4078b90 msgr2=0x7fe8f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:42.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.841+0000 7fe917656640 1 --2- 172.21.15.87:0/422937609 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe8f4078b90 0x7fe8f407b0a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe900000f60 tx=0x7fe900000ef0 comp rx=0 tx=0).stop 2024-09-17T03:14:42.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.841+0000 7fe917656640 1 -- 172.21.15.87:0/422937609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8fc004d40 msgr2=0x7fe9100b24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:42.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.841+0000 7fe917656640 1 --2- 172.21.15.87:0/422937609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8fc004d40 0x7fe9100b24f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe90c011a50 tx=0x7fe90c011f10 comp rx=0 tx=0).stop 2024-09-17T03:14:42.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.841+0000 7fe917656640 1 -- 172.21.15.87:0/422937609 shutdown_connections 2024-09-17T03:14:42.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.841+0000 7fe917656640 1 --2- 172.21.15.87:0/422937609 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe8f4078b90 0x7fe8f407b0a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:42.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.841+0000 7fe917656640 1 --2- 172.21.15.87:0/422937609 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe9101f7390 0x7fe9101f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:42.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.841+0000 7fe917656640 1 --2- 172.21.15.87:0/422937609 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe9100b2a30 0x7fe9101f6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:42.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.841+0000 7fe917656640 1 --2- 172.21.15.87:0/422937609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8fc004d40 0x7fe9100b24f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:42.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.841+0000 7fe917656640 1 -- 172.21.15.87:0/422937609 >> 172.21.15.87:0/422937609 conn(0x7fe910069360 msgr2=0x7fe9100d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:42.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.841+0000 7fe917656640 1 -- 172.21.15.87:0/422937609 shutdown_connections 2024-09-17T03:14:42.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:42.841+0000 7fe917656640 1 -- 172.21.15.87:0/422937609 wait complete. 2024-09-17T03:14:42.851 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi087.front.sepia.ceph.com... 2024-09-17T03:14:42.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:42.852 DEBUG:teuthology.orchestra.run.smithi087:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-09-17T03:14:42.876 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:14:42.877 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-09-17T03:14:42.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:42.877 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-17T03:14:42.932 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:14:42.933 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:14:42.952 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:14:42.952 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:14:43.011 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:14:43.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.124+0000 7fa9ecb0b640 1 -- 172.21.15.87:0/1186488733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9e80b7f10 msgr2=0x7fa9e815c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:43.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.124+0000 7fa9ecb0b640 1 --2- 172.21.15.87:0/1186488733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9e80b7f10 0x7fa9e815c5f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8005fd0 tx=0x7fa9d802a5a0 comp rx=0 tx=0).stop 2024-09-17T03:14:43.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.124+0000 7fa9ecb0b640 1 -- 172.21.15.87:0/1186488733 shutdown_connections 2024-09-17T03:14:43.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.124+0000 7fa9ecb0b640 1 --2- 172.21.15.87:0/1186488733 >> v2:172.21.15.149:3300/0 conn(0x7fa9e80d0230 0x7fa9e80d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:43.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.124+0000 7fa9ecb0b640 1 --2- 172.21.15.87:0/1186488733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9e80b7f10 0x7fa9e815c5f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:43.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.124+0000 7fa9ecb0b640 1 -- 172.21.15.87:0/1186488733 >> 172.21.15.87:0/1186488733 conn(0x7fa9e8069330 msgr2=0x7fa9e80ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:43.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.124+0000 7fa9ecb0b640 1 -- 172.21.15.87:0/1186488733 shutdown_connections 2024-09-17T03:14:43.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.124+0000 7fa9ecb0b640 1 -- 172.21.15.87:0/1186488733 wait complete. 2024-09-17T03:14:43.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.126+0000 7fa9ecb0b640 1 Processor -- start 2024-09-17T03:14:43.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.126+0000 7fa9ecb0b640 1 -- start start 2024-09-17T03:14:43.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.128+0000 7fa9ecb0b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9e80b3d60 0x7fa9e80a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:43.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.128+0000 7fa9ecb0b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9e80b7f10 0x7fa9e8155800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:43.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.128+0000 7fa9ecb0b640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9e80d0230 0x7fa9e80ac350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:43.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.128+0000 7fa9ecb0b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e80d0b80 con 0x7fa9e80b3d60 2024-09-17T03:14:43.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.128+0000 7fa9ecb0b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e80b7d20 con 0x7fa9e80b7f10 2024-09-17T03:14:43.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.128+0000 7fa9ecb0b640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e80d0d00 con 0x7fa9e80d0230 2024-09-17T03:14:43.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.128+0000 7fa9e659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9e80b3d60 0x7fa9e80a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:43.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.128+0000 7fa9e659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9e80b3d60 0x7fa9e80a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58366/0 (socket says 172.21.15.87:58366) 2024-09-17T03:14:43.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.128+0000 7fa9e659c640 1 -- 172.21.15.87:0/3915267561 learned_addr learned my addr 172.21.15.87:0/3915267561 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:43.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.128+0000 7fa9e5d9b640 1 --2- 172.21.15.87:0/3915267561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9e80b7f10 0x7fa9e8155800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:43.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.128+0000 7fa9e6d9d640 1 --2- 172.21.15.87:0/3915267561 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9e80d0230 0x7fa9e80ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:43.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.128+0000 7fa9e659c640 1 -- 172.21.15.87:0/3915267561 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9e80d0230 msgr2=0x7fa9e80ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:43.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.128+0000 7fa9e659c640 1 --2- 172.21.15.87:0/3915267561 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9e80d0230 0x7fa9e80ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:43.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.128+0000 7fa9e659c640 1 -- 172.21.15.87:0/3915267561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9e80b7f10 msgr2=0x7fa9e8155800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:43.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.128+0000 7fa9e659c640 1 --2- 172.21.15.87:0/3915267561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9e80b7f10 0x7fa9e8155800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:43.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.128+0000 7fa9e659c640 1 -- 172.21.15.87:0/3915267561 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e8155d40 con 0x7fa9e80b3d60 2024-09-17T03:14:43.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.128+0000 7fa9e659c640 1 --2- 172.21.15.87:0/3915267561 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9e80b3d60 0x7fa9e80a94a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8034040 tx=0x7fa9d80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:43.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.128+0000 7fa9d6ffd640 1 -- 172.21.15.87:0/3915267561 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d803e030 con 0x7fa9e80b3d60 2024-09-17T03:14:43.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.128+0000 7fa9d6ffd640 1 -- 172.21.15.87:0/3915267561 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9d8003d90 con 0x7fa9e80b3d60 2024-09-17T03:14:43.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.128+0000 7fa9ecb0b640 1 -- 172.21.15.87:0/3915267561 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e81f4e70 con 0x7fa9e80b3d60 2024-09-17T03:14:43.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.128+0000 7fa9d6ffd640 1 -- 172.21.15.87:0/3915267561 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8035040 con 0x7fa9e80b3d60 2024-09-17T03:14:43.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.128+0000 7fa9ecb0b640 1 -- 172.21.15.87:0/3915267561 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e81f5430 con 0x7fa9e80b3d60 2024-09-17T03:14:43.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.129+0000 7fa9ecb0b640 1 -- 172.21.15.87:0/3915267561 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e80d11d0 con 0x7fa9e80b3d60 2024-09-17T03:14:43.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.130+0000 7fa9d6ffd640 1 -- 172.21.15.87:0/3915267561 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa9d8041030 con 0x7fa9e80b3d60 2024-09-17T03:14:43.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.130+0000 7fa9d6ffd640 1 --2- 172.21.15.87:0/3915267561 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa9c8078b90 0x7fa9c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:43.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.131+0000 7fa9d6ffd640 1 -- 172.21.15.87:0/3915267561 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa9d80bb920 con 0x7fa9e80b3d60 2024-09-17T03:14:43.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.131+0000 7fa9e5d9b640 1 --2- 172.21.15.87:0/3915267561 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa9c8078b90 0x7fa9c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:43.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.131+0000 7fa9e5d9b640 1 --2- 172.21.15.87:0/3915267561 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa9c8078b90 0x7fa9c807b0a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8160040 tx=0x7fa9dc009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:43.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.135+0000 7fa9d6ffd640 1 -- 172.21.15.87:0/3915267561 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa9d8083ce0 con 0x7fa9e80b3d60 2024-09-17T03:14:43.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.320+0000 7fa9ecb0b640 1 -- 172.21.15.87:0/3915267561 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa9e815f2d0 con 0x7fa9e80b3d60 2024-09-17T03:14:43.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.320+0000 7fa9d6ffd640 1 -- 172.21.15.87:0/3915267561 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fa9d80835e0 con 0x7fa9e80b3d60 2024-09-17T03:14:43.320 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-09-17T03:14:43.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.322+0000 7fa9ecb0b640 1 -- 172.21.15.87:0/3915267561 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa9c8078b90 msgr2=0x7fa9c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:43.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.322+0000 7fa9ecb0b640 1 --2- 172.21.15.87:0/3915267561 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa9c8078b90 0x7fa9c807b0a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8160040 tx=0x7fa9dc009040 comp rx=0 tx=0).stop 2024-09-17T03:14:43.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.322+0000 7fa9ecb0b640 1 -- 172.21.15.87:0/3915267561 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9e80b3d60 msgr2=0x7fa9e80a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:43.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.322+0000 7fa9ecb0b640 1 --2- 172.21.15.87:0/3915267561 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9e80b3d60 0x7fa9e80a94a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8034040 tx=0x7fa9d80038a0 comp rx=0 tx=0).stop 2024-09-17T03:14:43.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.322+0000 7fa9ecb0b640 1 -- 172.21.15.87:0/3915267561 shutdown_connections 2024-09-17T03:14:43.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.322+0000 7fa9ecb0b640 1 --2- 172.21.15.87:0/3915267561 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa9c8078b90 0x7fa9c807b0a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:43.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.323+0000 7fa9ecb0b640 1 --2- 172.21.15.87:0/3915267561 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa9e80d0230 0x7fa9e80ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:43.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.323+0000 7fa9ecb0b640 1 --2- 172.21.15.87:0/3915267561 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9e80b7f10 0x7fa9e8155800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:43.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.323+0000 7fa9ecb0b640 1 --2- 172.21.15.87:0/3915267561 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9e80b3d60 0x7fa9e80a94a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:43.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.323+0000 7fa9ecb0b640 1 -- 172.21.15.87:0/3915267561 >> 172.21.15.87:0/3915267561 conn(0x7fa9e8069330 msgr2=0x7fa9e815b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:43.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.323+0000 7fa9ecb0b640 1 -- 172.21.15.87:0/3915267561 shutdown_connections 2024-09-17T03:14:43.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.323+0000 7fa9ecb0b640 1 -- 172.21.15.87:0/3915267561 wait complete. 2024-09-17T03:14:43.334 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:14:43.334 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:14:43.354 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:14:43.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.468+0000 7fe0b97da640 1 -- 172.21.15.87:0/1739295419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0b40d2c50 msgr2=0x7fe0b40d3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:43.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.468+0000 7fe0b97da640 1 --2- 172.21.15.87:0/1739295419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0b40d2c50 0x7fe0b40d3060 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4005fd0 tx=0x7fe0a402a820 comp rx=0 tx=0).stop 2024-09-17T03:14:43.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.468+0000 7fe0b97da640 1 -- 172.21.15.87:0/1739295419 shutdown_connections 2024-09-17T03:14:43.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.468+0000 7fe0b97da640 1 --2- 172.21.15.87:0/1739295419 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0b40d2c50 0x7fe0b40d3060 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:43.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.468+0000 7fe0b97da640 1 -- 172.21.15.87:0/1739295419 >> 172.21.15.87:0/1739295419 conn(0x7fe0b408dae0 msgr2=0x7fe0b409d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:43.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.468+0000 7fe0b97da640 1 -- 172.21.15.87:0/1739295419 shutdown_connections 2024-09-17T03:14:43.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.468+0000 7fe0b97da640 1 -- 172.21.15.87:0/1739295419 wait complete. 2024-09-17T03:14:43.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.470+0000 7fe0b97da640 1 Processor -- start 2024-09-17T03:14:43.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.470+0000 7fe0b97da640 1 -- start start 2024-09-17T03:14:43.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.471+0000 7fe0b97da640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0b40cefb0 0x7fe0b415a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:43.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.471+0000 7fe0b97da640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0b40d2c50 0x7fe0b40b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.471+0000 7fe0b97da640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe0b40d4150 0x7fe0b40b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.471+0000 7fe0b97da640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b40cfc90 con 0x7fe0b40cefb0 2024-09-17T03:14:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.471+0000 7fe0b97da640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe0b40cfb10 con 0x7fe0b40d2c50 2024-09-17T03:14:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.471+0000 7fe0b97da640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe0b40cfe10 con 0x7fe0b40d4150 2024-09-17T03:14:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.471+0000 7fe0b2ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0b40cefb0 0x7fe0b415a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.471+0000 7fe0b2ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0b40cefb0 0x7fe0b415a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58378/0 (socket says 172.21.15.87:58378) 2024-09-17T03:14:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.471+0000 7fe0b2ffd640 1 -- 172.21.15.87:0/427401550 learned_addr learned my addr 172.21.15.87:0/427401550 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.471+0000 7fe0b37fe640 1 --2- 172.21.15.87:0/427401550 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe0b40d4150 0x7fe0b40b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.471+0000 7fe0b27fc640 1 --2- 172.21.15.87:0/427401550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0b40d2c50 0x7fe0b40b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.471+0000 7fe0b2ffd640 1 -- 172.21.15.87:0/427401550 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe0b40d4150 msgr2=0x7fe0b40b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.471+0000 7fe0b2ffd640 1 --2- 172.21.15.87:0/427401550 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe0b40d4150 0x7fe0b40b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.471+0000 7fe0b2ffd640 1 -- 172.21.15.87:0/427401550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0b40d2c50 msgr2=0x7fe0b40b33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:43.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.471+0000 7fe0b2ffd640 1 --2- 172.21.15.87:0/427401550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0b40d2c50 0x7fe0b40b33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:43.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.471+0000 7fe0b2ffd640 1 -- 172.21.15.87:0/427401550 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0b40b1c60 con 0x7fe0b40cefb0 2024-09-17T03:14:43.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.472+0000 7fe0b2ffd640 1 --2- 172.21.15.87:0/427401550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0b40cefb0 0x7fe0b415a240 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009cd0 tx=0x7fe0a4032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:43.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.472+0000 7fe09b7fe640 1 -- 172.21.15.87:0/427401550 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4037040 con 0x7fe0b40cefb0 2024-09-17T03:14:43.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.472+0000 7fe09b7fe640 1 -- 172.21.15.87:0/427401550 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0a4042070 con 0x7fe0b40cefb0 2024-09-17T03:14:43.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.472+0000 7fe0b97da640 1 -- 172.21.15.87:0/427401550 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0b41e8be0 con 0x7fe0b40cefb0 2024-09-17T03:14:43.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.472+0000 7fe09b7fe640 1 -- 172.21.15.87:0/427401550 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4004280 con 0x7fe0b40cefb0 2024-09-17T03:14:43.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.472+0000 7fe0b97da640 1 -- 172.21.15.87:0/427401550 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0b41e91a0 con 0x7fe0b40cefb0 2024-09-17T03:14:43.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.473+0000 7fe0b97da640 1 -- 172.21.15.87:0/427401550 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0b40d3060 con 0x7fe0b40cefb0 2024-09-17T03:14:43.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.474+0000 7fe09b7fe640 1 -- 172.21.15.87:0/427401550 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe0a4004420 con 0x7fe0b40cefb0 2024-09-17T03:14:43.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.474+0000 7fe09b7fe640 1 --2- 172.21.15.87:0/427401550 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe0a0078b90 0x7fe0a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:43.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.474+0000 7fe09b7fe640 1 -- 172.21.15.87:0/427401550 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe0a40bbb70 con 0x7fe0b40cefb0 2024-09-17T03:14:43.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.474+0000 7fe0b27fc640 1 --2- 172.21.15.87:0/427401550 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe0a0078b90 0x7fe0a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:43.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.475+0000 7fe0b27fc640 1 --2- 172.21.15.87:0/427401550 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe0a0078b90 0x7fe0a007b0a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe0b41606e0 tx=0x7fe0a8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:43.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.479+0000 7fe09b7fe640 1 -- 172.21.15.87:0/427401550 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe0a40840d0 con 0x7fe0b40cefb0 2024-09-17T03:14:43.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.666+0000 7fe0b97da640 1 -- 172.21.15.87:0/427401550 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe0b40ce910 con 0x7fe0b40cefb0 2024-09-17T03:14:43.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.667+0000 7fe09b7fe640 1 -- 172.21.15.87:0/427401550 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fe0a40839d0 con 0x7fe0b40cefb0 2024-09-17T03:14:43.667 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-09-17T03:14:43.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.669+0000 7fe0b97da640 1 -- 172.21.15.87:0/427401550 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe0a0078b90 msgr2=0x7fe0a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:43.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.669+0000 7fe0b97da640 1 --2- 172.21.15.87:0/427401550 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe0a0078b90 0x7fe0a007b0a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe0b41606e0 tx=0x7fe0a8009040 comp rx=0 tx=0).stop 2024-09-17T03:14:43.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.669+0000 7fe0b97da640 1 -- 172.21.15.87:0/427401550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0b40cefb0 msgr2=0x7fe0b415a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:43.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.669+0000 7fe0b97da640 1 --2- 172.21.15.87:0/427401550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0b40cefb0 0x7fe0b415a240 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe0a4009cd0 tx=0x7fe0a4032d10 comp rx=0 tx=0).stop 2024-09-17T03:14:43.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.669+0000 7fe0b97da640 1 -- 172.21.15.87:0/427401550 shutdown_connections 2024-09-17T03:14:43.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.669+0000 7fe0b97da640 1 --2- 172.21.15.87:0/427401550 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe0a0078b90 0x7fe0a007b0a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:43.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.669+0000 7fe0b97da640 1 --2- 172.21.15.87:0/427401550 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe0b40d4150 0x7fe0b40b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:43.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.669+0000 7fe0b97da640 1 --2- 172.21.15.87:0/427401550 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe0b40d2c50 0x7fe0b40b33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:43.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.669+0000 7fe0b97da640 1 --2- 172.21.15.87:0/427401550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0b40cefb0 0x7fe0b415a240 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:43.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.669+0000 7fe0b97da640 1 -- 172.21.15.87:0/427401550 >> 172.21.15.87:0/427401550 conn(0x7fe0b408dae0 msgr2=0x7fe0b40d4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:43.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.669+0000 7fe0b97da640 1 -- 172.21.15.87:0/427401550 shutdown_connections 2024-09-17T03:14:43.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.669+0000 7fe0b97da640 1 -- 172.21.15.87:0/427401550 wait complete. 2024-09-17T03:14:43.680 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi149.front.sepia.ceph.com... 2024-09-17T03:14:43.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:43.680 DEBUG:teuthology.orchestra.run.smithi149:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-09-17T03:14:43.711 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:14:43.711 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi149.front.sepia.ceph.com 2024-09-17T03:14:43.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:14:43.711 DEBUG:teuthology.orchestra.run.smithi149:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-17T03:14:43.770 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T03:14:43.770 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:14:43.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.882+0000 7f299ddb0640 1 -- 172.21.15.87:0/1639138069 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29980b3d00 msgr2=0x7f29980b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:43.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.882+0000 7f299ddb0640 1 --2- 172.21.15.87:0/1639138069 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29980b3d00 0x7f29980b4110 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f298c009830 tx=0x7f298c02a480 comp rx=0 tx=0).stop 2024-09-17T03:14:43.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.882+0000 7f299ddb0640 1 -- 172.21.15.87:0/1639138069 shutdown_connections 2024-09-17T03:14:43.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.882+0000 7f299ddb0640 1 --2- 172.21.15.87:0/1639138069 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f29980d0460 0x7f29980d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:43.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.882+0000 7f299ddb0640 1 --2- 172.21.15.87:0/1639138069 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29980b3d00 0x7f29980b4110 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:43.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.882+0000 7f299ddb0640 1 -- 172.21.15.87:0/1639138069 >> 172.21.15.87:0/1639138069 conn(0x7f2998069360 msgr2=0x7f29980ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:43.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.882+0000 7f299ddb0640 1 -- 172.21.15.87:0/1639138069 shutdown_connections 2024-09-17T03:14:43.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.882+0000 7f299ddb0640 1 -- 172.21.15.87:0/1639138069 wait complete. 2024-09-17T03:14:43.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.884+0000 7f299ddb0640 1 Processor -- start 2024-09-17T03:14:43.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.884+0000 7f299ddb0640 1 -- start start 2024-09-17T03:14:43.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.886+0000 7f299ddb0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29980b3d00 0x7f29980b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:43.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.886+0000 7f299ddb0640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29980d0460 0x7f299806f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:43.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.886+0000 7f299ddb0640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f299815c6c0 0x7f299814d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:43.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.886+0000 7f299ddb0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f29980d0f30 con 0x7f29980b3d00 2024-09-17T03:14:43.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.886+0000 7f299ddb0640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f29980d0db0 con 0x7f29980d0460 2024-09-17T03:14:43.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.886+0000 7f299ddb0640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f29980d10b0 con 0x7f299815c6c0 2024-09-17T03:14:43.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.886+0000 7f29977fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29980b3d00 0x7f29980b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:43.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.886+0000 7f29977fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29980b3d00 0x7f29980b3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58392/0 (socket says 172.21.15.87:58392) 2024-09-17T03:14:43.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.886+0000 7f29977fe640 1 -- 172.21.15.87:0/3134579562 learned_addr learned my addr 172.21.15.87:0/3134579562 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:43.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.886+0000 7f2996ffd640 1 --2- 172.21.15.87:0/3134579562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29980d0460 0x7f299806f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:43.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.886+0000 7f2997fff640 1 --2- 172.21.15.87:0/3134579562 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f299815c6c0 0x7f299814d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:43.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.886+0000 7f29977fe640 1 -- 172.21.15.87:0/3134579562 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f299815c6c0 msgr2=0x7f299814d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:43.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.886+0000 7f29977fe640 1 --2- 172.21.15.87:0/3134579562 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f299815c6c0 0x7f299814d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:43.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.886+0000 7f29977fe640 1 -- 172.21.15.87:0/3134579562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29980d0460 msgr2=0x7f299806f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:43.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.886+0000 7f29977fe640 1 --2- 172.21.15.87:0/3134579562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29980d0460 0x7f299806f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:43.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.886+0000 7f29977fe640 1 -- 172.21.15.87:0/3134579562 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299806e900 con 0x7f29980b3d00 2024-09-17T03:14:43.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.886+0000 7f2996ffd640 1 --2- 172.21.15.87:0/3134579562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29980d0460 0x7f299806f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:14:43.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.886+0000 7f2997fff640 1 --2- 172.21.15.87:0/3134579562 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f299815c6c0 0x7f299814d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:14:43.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.886+0000 7f29977fe640 1 --2- 172.21.15.87:0/3134579562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29980b3d00 0x7f29980b3720 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f298800c3f0 tx=0x7f298800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:43.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.886+0000 7f297ffff640 1 -- 172.21.15.87:0/3134579562 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988017070 con 0x7f29980b3d00 2024-09-17T03:14:43.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.886+0000 7f297ffff640 1 -- 172.21.15.87:0/3134579562 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29880035f0 con 0x7f29980b3d00 2024-09-17T03:14:43.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.887+0000 7f299ddb0640 1 -- 172.21.15.87:0/3134579562 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2998160240 con 0x7f29980b3d00 2024-09-17T03:14:43.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.887+0000 7f297ffff640 1 -- 172.21.15.87:0/3134579562 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988003a40 con 0x7f29980b3d00 2024-09-17T03:14:43.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.887+0000 7f299ddb0640 1 -- 172.21.15.87:0/3134579562 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f299806eea0 con 0x7f29980b3d00 2024-09-17T03:14:43.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.887+0000 7f299ddb0640 1 -- 172.21.15.87:0/3134579562 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29980d1640 con 0x7f29980b3d00 2024-09-17T03:14:43.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.888+0000 7f297ffff640 1 -- 172.21.15.87:0/3134579562 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f298801a070 con 0x7f29980b3d00 2024-09-17T03:14:43.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.889+0000 7f297ffff640 1 --2- 172.21.15.87:0/3134579562 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f2984078b90 0x7f298407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:43.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.889+0000 7f297ffff640 1 -- 172.21.15.87:0/3134579562 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f298809a0b0 con 0x7f29980b3d00 2024-09-17T03:14:43.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.889+0000 7f2996ffd640 1 --2- 172.21.15.87:0/3134579562 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f2984078b90 0x7f298407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:43.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.890+0000 7f2996ffd640 1 --2- 172.21.15.87:0/3134579562 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f2984078b90 0x7f298407b0a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f298c034000 tx=0x7f298c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:43.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:43.893+0000 7f297ffff640 1 -- 172.21.15.87:0/3134579562 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f29980b4650 con 0x7f29980b3d00 2024-09-17T03:14:44.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.102+0000 7f299ddb0640 1 -- 172.21.15.87:0/3134579562 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29980b6690 con 0x7f29980b3d00 2024-09-17T03:14:44.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.103+0000 7f297ffff640 1 -- 172.21.15.87:0/3134579562 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5217 (secure 0 0 0) 0x7f29880624f0 con 0x7f29980b3d00 2024-09-17T03:14:44.103 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:44.103 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":6,"btime":"2024-09-17T03:14:16:330267+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4289,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/110106418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":110106418},{"type":"v1","addr":"172.21.15.149:6835","nonce":110106418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1336626873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1336626873},{"type":"v1","addr":"172.21.15.149:6837","nonce":1336626873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/666860906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":666860906},{"type":"v1","addr":"172.21.15.87:6833","nonce":666860906}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:14:13.672621+0000","modified":"2024-09-17T03:14:15.675568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4398":{"gid":4398,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/1831073338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1831073338},{"type":"v1","addr":"172.21.15.87:6835","nonce":1831073338}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4398,"qdb_cluster":[4398]},"id":1}]} 2024-09-17T03:14:44.103 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 6 2024-09-17T03:14:44.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.105+0000 7f299ddb0640 1 -- 172.21.15.87:0/3134579562 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f2984078b90 msgr2=0x7f298407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:44.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.105+0000 7f299ddb0640 1 --2- 172.21.15.87:0/3134579562 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f2984078b90 0x7f298407b0a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f298c034000 tx=0x7f298c004730 comp rx=0 tx=0).stop 2024-09-17T03:14:44.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.105+0000 7f299ddb0640 1 -- 172.21.15.87:0/3134579562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29980b3d00 msgr2=0x7f29980b3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:44.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.105+0000 7f299ddb0640 1 --2- 172.21.15.87:0/3134579562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29980b3d00 0x7f29980b3720 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f298800c3f0 tx=0x7f298800c8b0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.105+0000 7f299ddb0640 1 -- 172.21.15.87:0/3134579562 shutdown_connections 2024-09-17T03:14:44.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.105+0000 7f299ddb0640 1 --2- 172.21.15.87:0/3134579562 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f2984078b90 0x7f298407b0a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.105+0000 7f299ddb0640 1 --2- 172.21.15.87:0/3134579562 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f299815c6c0 0x7f299814d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.105+0000 7f299ddb0640 1 --2- 172.21.15.87:0/3134579562 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f29980d0460 0x7f299806f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.105+0000 7f299ddb0640 1 --2- 172.21.15.87:0/3134579562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29980b3d00 0x7f29980b3720 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.105+0000 7f299ddb0640 1 -- 172.21.15.87:0/3134579562 >> 172.21.15.87:0/3134579562 conn(0x7f2998069360 msgr2=0x7f299815b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:44.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.105+0000 7f299ddb0640 1 -- 172.21.15.87:0/3134579562 shutdown_connections 2024-09-17T03:14:44.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.105+0000 7f299ddb0640 1 -- 172.21.15.87:0/3134579562 wait complete. 2024-09-17T03:14:44.116 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:14:44.228 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.228+0000 7fd870beb640 1 -- 172.21.15.87:0/290837677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c0cc590 msgr2=0x7fd86c15c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:44.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.228+0000 7fd870beb640 1 --2- 172.21.15.87:0/290837677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c0cc590 0x7fd86c15c470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd86c072ca0 tx=0x7fd85c02a560 comp rx=0 tx=0).stop 2024-09-17T03:14:44.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.228+0000 7fd870beb640 1 -- 172.21.15.87:0/290837677 shutdown_connections 2024-09-17T03:14:44.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.228+0000 7fd870beb640 1 --2- 172.21.15.87:0/290837677 >> v2:172.21.15.149:3300/0 conn(0x7fd86c15f4d0 0x7fd86c074de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.228+0000 7fd870beb640 1 --2- 172.21.15.87:0/290837677 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd86c15f0c0 0x7fd86c0748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.228+0000 7fd870beb640 1 --2- 172.21.15.87:0/290837677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c0cc590 0x7fd86c15c470 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.228+0000 7fd870beb640 1 -- 172.21.15.87:0/290837677 >> 172.21.15.87:0/290837677 conn(0x7fd86c06ee70 msgr2=0x7fd86c06f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:44.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.228+0000 7fd870beb640 1 -- 172.21.15.87:0/290837677 shutdown_connections 2024-09-17T03:14:44.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.228+0000 7fd870beb640 1 -- 172.21.15.87:0/290837677 wait complete. 2024-09-17T03:14:44.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.230+0000 7fd870beb640 1 Processor -- start 2024-09-17T03:14:44.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.230+0000 7fd870beb640 1 -- start start 2024-09-17T03:14:44.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.232+0000 7fd870beb640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c0cc590 0x7fd86c148e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:44.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.232+0000 7fd870beb640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd86c15f0c0 0x7fd86c1f3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:44.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.232+0000 7fd870beb640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd86c15f4d0 0x7fd86c1f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:44.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.232+0000 7fd870beb640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd86c0c4f00 con 0x7fd86c0cc590 2024-09-17T03:14:44.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.232+0000 7fd870beb640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd86c0c6b50 con 0x7fd86c15f0c0 2024-09-17T03:14:44.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.232+0000 7fd870beb640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd86c0c5080 con 0x7fd86c15f4d0 2024-09-17T03:14:44.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.232+0000 7fd86a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c0cc590 0x7fd86c148e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:44.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.232+0000 7fd86a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c0cc590 0x7fd86c148e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58406/0 (socket says 172.21.15.87:58406) 2024-09-17T03:14:44.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.232+0000 7fd86a59c640 1 -- 172.21.15.87:0/2271104610 learned_addr learned my addr 172.21.15.87:0/2271104610 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:44.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.232+0000 7fd86ad9d640 1 --2- 172.21.15.87:0/2271104610 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd86c15f4d0 0x7fd86c1f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:44.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.232+0000 7fd869d9b640 1 --2- 172.21.15.87:0/2271104610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd86c15f0c0 0x7fd86c1f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:44.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.232+0000 7fd86a59c640 1 -- 172.21.15.87:0/2271104610 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd86c15f4d0 msgr2=0x7fd86c1f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:44.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.232+0000 7fd86a59c640 1 --2- 172.21.15.87:0/2271104610 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd86c15f4d0 0x7fd86c1f3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.232+0000 7fd86a59c640 1 -- 172.21.15.87:0/2271104610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd86c15f0c0 msgr2=0x7fd86c1f3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:44.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.232+0000 7fd86a59c640 1 --2- 172.21.15.87:0/2271104610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd86c15f0c0 0x7fd86c1f3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.232+0000 7fd86a59c640 1 -- 172.21.15.87:0/2271104610 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd86c1f6cb0 con 0x7fd86c0cc590 2024-09-17T03:14:44.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.232+0000 7fd86a59c640 1 --2- 172.21.15.87:0/2271104610 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c0cc590 0x7fd86c148e10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd85c034040 tx=0x7fd85c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:44.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.232+0000 7fd85affd640 1 -- 172.21.15.87:0/2271104610 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c03e030 con 0x7fd86c0cc590 2024-09-17T03:14:44.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.232+0000 7fd85affd640 1 -- 172.21.15.87:0/2271104610 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd85c0047a0 con 0x7fd86c0cc590 2024-09-17T03:14:44.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.232+0000 7fd870beb640 1 -- 172.21.15.87:0/2271104610 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd86c1f6f90 con 0x7fd86c0cc590 2024-09-17T03:14:44.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.232+0000 7fd85affd640 1 -- 172.21.15.87:0/2271104610 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd85c035040 con 0x7fd86c0cc590 2024-09-17T03:14:44.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.233+0000 7fd870beb640 1 -- 172.21.15.87:0/2271104610 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd86c1f7550 con 0x7fd86c0cc590 2024-09-17T03:14:44.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.233+0000 7fd870beb640 1 -- 172.21.15.87:0/2271104610 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd86c15c470 con 0x7fd86c0cc590 2024-09-17T03:14:44.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.234+0000 7fd85affd640 1 -- 172.21.15.87:0/2271104610 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd85c041030 con 0x7fd86c0cc590 2024-09-17T03:14:44.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.235+0000 7fd85affd640 1 --2- 172.21.15.87:0/2271104610 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd84c078b90 0x7fd84c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:44.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.235+0000 7fd85affd640 1 -- 172.21.15.87:0/2271104610 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd85c0bb650 con 0x7fd86c0cc590 2024-09-17T03:14:44.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.235+0000 7fd869d9b640 1 --2- 172.21.15.87:0/2271104610 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd84c078b90 0x7fd84c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:44.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.235+0000 7fd869d9b640 1 --2- 172.21.15.87:0/2271104610 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd84c078b90 0x7fd84c07b0a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd860005fd0 tx=0x7fd860005d80 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:44.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.239+0000 7fd85affd640 1 -- 172.21.15.87:0/2271104610 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd85c083b30 con 0x7fd86c0cc590 2024-09-17T03:14:44.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.447+0000 7fd870beb640 1 -- 172.21.15.87:0/2271104610 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd86c159070 con 0x7fd86c0cc590 2024-09-17T03:14:44.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.448+0000 7fd85affd640 1 -- 172.21.15.87:0/2271104610 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5217 (secure 0 0 0) 0x7fd85c083430 con 0x7fd86c0cc590 2024-09-17T03:14:44.448 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:44.448 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":6,"btime":"2024-09-17T03:14:16:330267+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4289,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/110106418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":110106418},{"type":"v1","addr":"172.21.15.149:6835","nonce":110106418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1336626873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1336626873},{"type":"v1","addr":"172.21.15.149:6837","nonce":1336626873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/666860906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":666860906},{"type":"v1","addr":"172.21.15.87:6833","nonce":666860906}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:14:13.672621+0000","modified":"2024-09-17T03:14:15.675568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4398":{"gid":4398,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/1831073338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1831073338},{"type":"v1","addr":"172.21.15.87:6835","nonce":1831073338}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4398,"qdb_cluster":[4398]},"id":1}]} 2024-09-17T03:14:44.448 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 6 2024-09-17T03:14:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.450+0000 7fd870beb640 1 -- 172.21.15.87:0/2271104610 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd84c078b90 msgr2=0x7fd84c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.450+0000 7fd870beb640 1 --2- 172.21.15.87:0/2271104610 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd84c078b90 0x7fd84c07b0a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd860005fd0 tx=0x7fd860005d80 comp rx=0 tx=0).stop 2024-09-17T03:14:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.450+0000 7fd870beb640 1 -- 172.21.15.87:0/2271104610 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c0cc590 msgr2=0x7fd86c148e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.450+0000 7fd870beb640 1 --2- 172.21.15.87:0/2271104610 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c0cc590 0x7fd86c148e10 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd85c034040 tx=0x7fd85c0039e0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.450+0000 7fd870beb640 1 -- 172.21.15.87:0/2271104610 shutdown_connections 2024-09-17T03:14:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.450+0000 7fd870beb640 1 --2- 172.21.15.87:0/2271104610 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd84c078b90 0x7fd84c07b0a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.450+0000 7fd870beb640 1 --2- 172.21.15.87:0/2271104610 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd86c15f4d0 0x7fd86c1f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.450+0000 7fd870beb640 1 --2- 172.21.15.87:0/2271104610 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd86c15f0c0 0x7fd86c1f3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.450+0000 7fd870beb640 1 --2- 172.21.15.87:0/2271104610 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd86c0cc590 0x7fd86c148e10 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.450+0000 7fd870beb640 1 -- 172.21.15.87:0/2271104610 >> 172.21.15.87:0/2271104610 conn(0x7fd86c06ee70 msgr2=0x7fd86c072c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:44.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.450+0000 7fd870beb640 1 -- 172.21.15.87:0/2271104610 shutdown_connections 2024-09-17T03:14:44.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.450+0000 7fd870beb640 1 -- 172.21.15.87:0/2271104610 wait complete. 2024-09-17T03:14:44.461 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:14:44.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.575+0000 7f185e519640 1 -- 172.21.15.87:0/1867194215 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1840004d40 msgr2=0x7f1840005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:44.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.575+0000 7f185e519640 1 --2- 172.21.15.87:0/1867194215 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1840004d40 0x7f1840005190 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1850009e20 tx=0x7f1840005830 comp rx=0 tx=0).stop 2024-09-17T03:14:44.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.575+0000 7f185cab6640 1 -- 172.21.15.87:0/1867194215 reap_dead start 2024-09-17T03:14:44.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.576+0000 7f185e519640 1 -- 172.21.15.87:0/1867194215 shutdown_connections 2024-09-17T03:14:44.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.576+0000 7f185e519640 1 -- 172.21.15.87:0/1867194215 >> 172.21.15.87:0/1867194215 conn(0x7f1858069330 msgr2=0x7f18580ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:44.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.576+0000 7f185e519640 1 -- 172.21.15.87:0/1867194215 shutdown_connections 2024-09-17T03:14:44.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.576+0000 7f185e519640 1 -- 172.21.15.87:0/1867194215 wait complete. 2024-09-17T03:14:44.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.577+0000 7f185e519640 1 Processor -- start 2024-09-17T03:14:44.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.578+0000 7f185e519640 1 -- start start 2024-09-17T03:14:44.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.579+0000 7f185e519640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1840004d40 0x7f1858156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:44.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.579+0000 7f185e519640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1858156fd0 0x7f18581f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:44.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.579+0000 7f185e519640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f18581573e0 0x7f18581f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:44.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.579+0000 7f185e519640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f18580b7c90 con 0x7f1840004d40 2024-09-17T03:14:44.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.579+0000 7f185e519640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f18580b7b10 con 0x7f1858156fd0 2024-09-17T03:14:44.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.579+0000 7f185e519640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f185815c5f0 con 0x7f18581573e0 2024-09-17T03:14:44.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.579+0000 7f1857fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1840004d40 0x7f1858156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:44.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.579+0000 7f1857fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1840004d40 0x7f1858156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58420/0 (socket says 172.21.15.87:58420) 2024-09-17T03:14:44.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.579+0000 7f1857fff640 1 -- 172.21.15.87:0/1964016607 learned_addr learned my addr 172.21.15.87:0/1964016607 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:44.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.579+0000 7f1857fff640 1 -- 172.21.15.87:0/1964016607 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f18581573e0 msgr2=0x7f18581f3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:44.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.579+0000 7f1857fff640 1 --2- 172.21.15.87:0/1964016607 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f18581573e0 0x7f18581f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.579+0000 7f1857fff640 1 -- 172.21.15.87:0/1964016607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1858156fd0 msgr2=0x7f18581f31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:44.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.579+0000 7f1857fff640 1 --2- 172.21.15.87:0/1964016607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1858156fd0 0x7f18581f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.579+0000 7f1857fff640 1 -- 172.21.15.87:0/1964016607 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1858155490 con 0x7f1840004d40 2024-09-17T03:14:44.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.579+0000 7f1857fff640 1 --2- 172.21.15.87:0/1964016607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1840004d40 0x7f1858156250 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1848004600 tx=0x7f184800ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:44.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.579+0000 7f1854ff9640 1 -- 172.21.15.87:0/1964016607 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848010a60 con 0x7f1840004d40 2024-09-17T03:14:44.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.579+0000 7f1854ff9640 1 -- 172.21.15.87:0/1964016607 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1848015070 con 0x7f1840004d40 2024-09-17T03:14:44.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.579+0000 7f1854ff9640 1 -- 172.21.15.87:0/1964016607 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1848010e80 con 0x7f1840004d40 2024-09-17T03:14:44.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.580+0000 7f185e519640 1 -- 172.21.15.87:0/1964016607 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1858155770 con 0x7f1840004d40 2024-09-17T03:14:44.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.580+0000 7f185e519640 1 -- 172.21.15.87:0/1964016607 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1858156790 con 0x7f1840004d40 2024-09-17T03:14:44.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.581+0000 7f185e519640 1 -- 172.21.15.87:0/1964016607 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18580b4170 con 0x7f1840004d40 2024-09-17T03:14:44.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.581+0000 7f1854ff9640 1 -- 172.21.15.87:0/1964016607 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f184801a470 con 0x7f1840004d40 2024-09-17T03:14:44.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.582+0000 7f1854ff9640 1 --2- 172.21.15.87:0/1964016607 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1844078b90 0x7f184407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:44.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.582+0000 7f1854ff9640 1 -- 172.21.15.87:0/1964016607 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f184809b890 con 0x7f1840004d40 2024-09-17T03:14:44.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.582+0000 7f18577fe640 1 --2- 172.21.15.87:0/1964016607 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1844078b90 0x7f184407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:44.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.583+0000 7f18577fe640 1 --2- 172.21.15.87:0/1964016607 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1844078b90 0x7f184407b0a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f184c001d40 tx=0x7f184c001cb0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:44.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.586+0000 7f1854ff9640 1 -- 172.21.15.87:0/1964016607 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1848063d70 con 0x7f1840004d40 2024-09-17T03:14:44.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.761+0000 7f185e519640 1 -- 172.21.15.87:0/1964016607 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18580b6540 con 0x7f1840004d40 2024-09-17T03:14:44.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.762+0000 7f1854ff9640 1 -- 172.21.15.87:0/1964016607 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16693 (secure 0 0 0) 0x7f184800de50 con 0x7f1840004d40 2024-09-17T03:14:44.762 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:44.762 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":23,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:14:15.323716+0000","last_up_change":"2024-09-17T03:14:01.833605+0000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:14:03.433404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:14:11.525547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T03:14:12.626396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6825","nonce":2430409749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6827","nonce":2430409749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6831","nonce":2430409749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6829","nonce":2430409749}]},"public_addr":"172.21.15.87:6825/2430409749","cluster_addr":"172.21.15.87:6827/2430409749","heartbeat_back_addr":"172.21.15.87:6831/2430409749","heartbeat_front_addr":"172.21.15.87:6829/2430409749","state":["exists","up"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6809","nonce":3467605435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6811","nonce":3467605435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6815","nonce":3467605435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6813","nonce":3467605435}]},"public_addr":"172.21.15.87:6809/3467605435","cluster_addr":"172.21.15.87:6811/3467605435","heartbeat_back_addr":"172.21.15.87:6815/3467605435","heartbeat_front_addr":"172.21.15.87:6813/3467605435","state":["exists","up"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6801","nonce":1450834766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6803","nonce":1450834766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6807","nonce":1450834766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6805","nonce":1450834766}]},"public_addr":"172.21.15.87:6801/1450834766","cluster_addr":"172.21.15.87:6803/1450834766","heartbeat_back_addr":"172.21.15.87:6807/1450834766","heartbeat_front_addr":"172.21.15.87:6805/1450834766","state":["exists","up"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6817","nonce":4253161216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6819","nonce":4253161216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6823","nonce":4253161216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6821","nonce":4253161216}]},"public_addr":"172.21.15.87:6817/4253161216","cluster_addr":"172.21.15.87:6819/4253161216","heartbeat_back_addr":"172.21.15.87:6823/4253161216","heartbeat_front_addr":"172.21.15.87:6821/4253161216","state":["exists","up"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6803","nonce":2414907089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6805","nonce":2414907089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6809","nonce":2414907089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6807","nonce":2414907089}]},"public_addr":"172.21.15.149:6803/2414907089","cluster_addr":"172.21.15.149:6805/2414907089","heartbeat_back_addr":"172.21.15.149:6809/2414907089","heartbeat_front_addr":"172.21.15.149:6807/2414907089","state":["exists","up"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6827","nonce":2182999530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6829","nonce":2182999530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6833","nonce":2182999530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6831","nonce":2182999530}]},"public_addr":"172.21.15.149:6827/2182999530","cluster_addr":"172.21.15.149:6829/2182999530","heartbeat_back_addr":"172.21.15.149:6833/2182999530","heartbeat_front_addr":"172.21.15.149:6831/2182999530","state":["exists","up"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6811","nonce":4289519146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6813","nonce":4289519146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6817","nonce":4289519146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6815","nonce":4289519146}]},"public_addr":"172.21.15.149:6811/4289519146","cluster_addr":"172.21.15.149:6813/4289519146","heartbeat_back_addr":"172.21.15.149:6817/4289519146","heartbeat_front_addr":"172.21.15.149:6815/4289519146","state":["exists","up"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6819","nonce":2771010477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6821","nonce":2771010477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6825","nonce":2771010477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6823","nonce":2771010477}]},"public_addr":"172.21.15.149:6819/2771010477","cluster_addr":"172.21.15.149:6821/2771010477","heartbeat_back_addr":"172.21.15.149:6825/2771010477","heartbeat_front_addr":"172.21.15.149:6823/2771010477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.313297+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.642861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.654781+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.302606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.385791+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.915266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.121365+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.394432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:14:44.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.764+0000 7f185e519640 1 -- 172.21.15.87:0/1964016607 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1844078b90 msgr2=0x7f184407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:44.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.764+0000 7f185e519640 1 --2- 172.21.15.87:0/1964016607 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1844078b90 0x7f184407b0a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f184c001d40 tx=0x7f184c001cb0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.764+0000 7f185e519640 1 -- 172.21.15.87:0/1964016607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1840004d40 msgr2=0x7f1858156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:44.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.764+0000 7f185e519640 1 --2- 172.21.15.87:0/1964016607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1840004d40 0x7f1858156250 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1848004600 tx=0x7f184800ba40 comp rx=0 tx=0).stop 2024-09-17T03:14:44.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.764+0000 7f185e519640 1 -- 172.21.15.87:0/1964016607 shutdown_connections 2024-09-17T03:14:44.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.764+0000 7f185e519640 1 --2- 172.21.15.87:0/1964016607 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1844078b90 0x7f184407b0a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.764+0000 7f185e519640 1 --2- 172.21.15.87:0/1964016607 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f18581573e0 0x7f18581f3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.764+0000 7f185e519640 1 --2- 172.21.15.87:0/1964016607 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1858156fd0 0x7f18581f31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.764+0000 7f185e519640 1 --2- 172.21.15.87:0/1964016607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1840004d40 0x7f1858156250 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.764+0000 7f185e519640 1 -- 172.21.15.87:0/1964016607 >> 172.21.15.87:0/1964016607 conn(0x7f1858069330 msgr2=0x7f18580b6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:44.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.764+0000 7f185e519640 1 -- 172.21.15.87:0/1964016607 shutdown_connections 2024-09-17T03:14:44.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.764+0000 7f185e519640 1 -- 172.21.15.87:0/1964016607 wait complete. 2024-09-17T03:14:44.776 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T03:14:44.776 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:14:44.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.889+0000 7f87c2dda640 1 -- 172.21.15.87:0/12115887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87bc0d0230 msgr2=0x7f87bc0d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:44.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.889+0000 7f87c2dda640 1 --2- 172.21.15.87:0/12115887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87bc0d0230 0x7f87bc0d0640 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f87b4005fd0 tx=0x7f87b402a540 comp rx=0 tx=0).stop 2024-09-17T03:14:44.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.889+0000 7f87c2dda640 1 -- 172.21.15.87:0/12115887 shutdown_connections 2024-09-17T03:14:44.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.889+0000 7f87c2dda640 1 --2- 172.21.15.87:0/12115887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87bc0d0230 0x7f87bc0d0640 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.889+0000 7f87c2dda640 1 --2- 172.21.15.87:0/12115887 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f87bc0b4470 0x7f87bc15c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.889+0000 7f87c2dda640 1 -- 172.21.15.87:0/12115887 >> 172.21.15.87:0/12115887 conn(0x7f87bc069330 msgr2=0x7f87bc0ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:44.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.889+0000 7f87c2dda640 1 -- 172.21.15.87:0/12115887 shutdown_connections 2024-09-17T03:14:44.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.889+0000 7f87c2dda640 1 -- 172.21.15.87:0/12115887 wait complete. 2024-09-17T03:14:44.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.891+0000 7f87c2dda640 1 Processor -- start 2024-09-17T03:14:44.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.891+0000 7f87c2dda640 1 -- start start 2024-09-17T03:14:44.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.892+0000 7f87c2dda640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87bc0b7f10 0x7f87bc14b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:44.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.892+0000 7f87c2dda640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f87bc0d0230 0x7f87bc14b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:44.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.892+0000 7f87c2dda640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87bc0cb7b0 0x7f87bc14be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:44.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.892+0000 7f87c2dda640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f87bc0d0b80 con 0x7f87bc0cb7b0 2024-09-17T03:14:44.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.892+0000 7f87c2dda640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f87bc0b7d20 con 0x7f87bc0b7f10 2024-09-17T03:14:44.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.892+0000 7f87c2dda640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f87bc0d0d00 con 0x7f87bc0d0230 2024-09-17T03:14:44.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.892+0000 7f87c1377640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87bc0cb7b0 0x7f87bc14be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:44.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.892+0000 7f87c1377640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87bc0cb7b0 0x7f87bc14be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58428/0 (socket says 172.21.15.87:58428) 2024-09-17T03:14:44.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.892+0000 7f87c1377640 1 -- 172.21.15.87:0/1932446913 learned_addr learned my addr 172.21.15.87:0/1932446913 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:44.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.892+0000 7f87bbfff640 1 --2- 172.21.15.87:0/1932446913 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f87bc0d0230 0x7f87bc14b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:44.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.892+0000 7f87c0b76640 1 --2- 172.21.15.87:0/1932446913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87bc0b7f10 0x7f87bc14b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:44.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.893+0000 7f87c1377640 1 -- 172.21.15.87:0/1932446913 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f87bc0d0230 msgr2=0x7f87bc14b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:44.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.893+0000 7f87c1377640 1 --2- 172.21.15.87:0/1932446913 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f87bc0d0230 0x7f87bc14b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.893+0000 7f87c1377640 1 -- 172.21.15.87:0/1932446913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87bc0b7f10 msgr2=0x7f87bc14b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:44.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.893+0000 7f87c1377640 1 --2- 172.21.15.87:0/1932446913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87bc0b7f10 0x7f87bc14b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:44.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.893+0000 7f87c1377640 1 -- 172.21.15.87:0/1932446913 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87bc0ca710 con 0x7f87bc0cb7b0 2024-09-17T03:14:44.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.893+0000 7f87c0b76640 1 --2- 172.21.15.87:0/1932446913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87bc0b7f10 0x7f87bc14b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:44.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.893+0000 7f87bbfff640 1 --2- 172.21.15.87:0/1932446913 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f87bc0d0230 0x7f87bc14b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:44.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.893+0000 7f87c1377640 1 --2- 172.21.15.87:0/1932446913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87bc0cb7b0 0x7f87bc14be20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f87b4034040 tx=0x7f87b4003880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:44.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.893+0000 7f87b97fa640 1 -- 172.21.15.87:0/1932446913 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b403e030 con 0x7f87bc0cb7b0 2024-09-17T03:14:44.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.893+0000 7f87b97fa640 1 -- 172.21.15.87:0/1932446913 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87b4003d70 con 0x7f87bc0cb7b0 2024-09-17T03:14:44.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.893+0000 7f87c2dda640 1 -- 172.21.15.87:0/1932446913 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87bc0ca9f0 con 0x7f87bc0cb7b0 2024-09-17T03:14:44.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.893+0000 7f87b97fa640 1 -- 172.21.15.87:0/1932446913 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b4035040 con 0x7f87bc0cb7b0 2024-09-17T03:14:44.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.893+0000 7f87c2dda640 1 -- 172.21.15.87:0/1932446913 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87bc0cafb0 con 0x7f87bc0cb7b0 2024-09-17T03:14:44.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.894+0000 7f87c2dda640 1 -- 172.21.15.87:0/1932446913 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87bc069c90 con 0x7f87bc0cb7b0 2024-09-17T03:14:44.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.895+0000 7f87b97fa640 1 -- 172.21.15.87:0/1932446913 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f87b4041030 con 0x7f87bc0cb7b0 2024-09-17T03:14:44.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.895+0000 7f87b97fa640 1 --2- 172.21.15.87:0/1932446913 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f87a8078b90 0x7f87a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:44.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.895+0000 7f87b97fa640 1 -- 172.21.15.87:0/1932446913 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f87b40bb490 con 0x7f87bc0cb7b0 2024-09-17T03:14:44.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.895+0000 7f87c0b76640 1 --2- 172.21.15.87:0/1932446913 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f87a8078b90 0x7f87a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:44.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.896+0000 7f87c0b76640 1 --2- 172.21.15.87:0/1932446913 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f87a8078b90 0x7f87a807b0a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f87ac000f60 tx=0x7f87ac009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:44.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:44.900+0000 7f87b97fa640 1 -- 172.21.15.87:0/1932446913 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f87b4083960 con 0x7f87bc0cb7b0 2024-09-17T03:14:45.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.109+0000 7f87c2dda640 1 -- 172.21.15.87:0/1932446913 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87bc0d11d0 con 0x7f87bc0cb7b0 2024-09-17T03:14:45.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.110+0000 7f87b97fa640 1 -- 172.21.15.87:0/1932446913 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5217 (secure 0 0 0) 0x7f87b4083260 con 0x7f87bc0cb7b0 2024-09-17T03:14:45.110 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:45.110 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":6,"btime":"2024-09-17T03:14:16:330267+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4289,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/110106418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":110106418},{"type":"v1","addr":"172.21.15.149:6835","nonce":110106418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1336626873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1336626873},{"type":"v1","addr":"172.21.15.149:6837","nonce":1336626873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/666860906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":666860906},{"type":"v1","addr":"172.21.15.87:6833","nonce":666860906}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:14:13.672621+0000","modified":"2024-09-17T03:14:15.675568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4398":{"gid":4398,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/1831073338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1831073338},{"type":"v1","addr":"172.21.15.87:6835","nonce":1831073338}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4398,"qdb_cluster":[4398]},"id":1}]} 2024-09-17T03:14:45.110 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 6 2024-09-17T03:14:45.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.112+0000 7f87c2dda640 1 -- 172.21.15.87:0/1932446913 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f87a8078b90 msgr2=0x7f87a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:45.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.112+0000 7f87c2dda640 1 --2- 172.21.15.87:0/1932446913 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f87a8078b90 0x7f87a807b0a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f87ac000f60 tx=0x7f87ac009040 comp rx=0 tx=0).stop 2024-09-17T03:14:45.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.112+0000 7f87c2dda640 1 -- 172.21.15.87:0/1932446913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87bc0cb7b0 msgr2=0x7f87bc14be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:45.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.112+0000 7f87c2dda640 1 --2- 172.21.15.87:0/1932446913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87bc0cb7b0 0x7f87bc14be20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f87b4034040 tx=0x7f87b4003880 comp rx=0 tx=0).stop 2024-09-17T03:14:45.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.112+0000 7f87c2dda640 1 -- 172.21.15.87:0/1932446913 shutdown_connections 2024-09-17T03:14:45.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.112+0000 7f87c2dda640 1 --2- 172.21.15.87:0/1932446913 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f87a8078b90 0x7f87a807b0a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:45.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.112+0000 7f87c2dda640 1 --2- 172.21.15.87:0/1932446913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87bc0cb7b0 0x7f87bc14be20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:45.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.112+0000 7f87c2dda640 1 --2- 172.21.15.87:0/1932446913 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f87bc0d0230 0x7f87bc14b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:45.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.112+0000 7f87c2dda640 1 --2- 172.21.15.87:0/1932446913 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f87bc0b7f10 0x7f87bc14b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:45.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.112+0000 7f87c2dda640 1 -- 172.21.15.87:0/1932446913 >> 172.21.15.87:0/1932446913 conn(0x7f87bc069330 msgr2=0x7f87bc0b62c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:45.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.112+0000 7f87c2dda640 1 -- 172.21.15.87:0/1932446913 shutdown_connections 2024-09-17T03:14:45.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.112+0000 7f87c2dda640 1 -- 172.21.15.87:0/1932446913 wait complete. 2024-09-17T03:14:45.123 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:14:45.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.235+0000 7f3911952640 1 -- 172.21.15.87:0/4285371571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f390c15b490 msgr2=0x7f390c15c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:45.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.235+0000 7f3911952640 1 --2- 172.21.15.87:0/4285371571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f390c15b490 0x7f390c15c220 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f390c06ad50 tx=0x7f390002a580 comp rx=0 tx=0).stop 2024-09-17T03:14:45.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.235+0000 7f3911952640 1 -- 172.21.15.87:0/4285371571 shutdown_connections 2024-09-17T03:14:45.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.235+0000 7f3911952640 1 --2- 172.21.15.87:0/4285371571 >> v2:172.21.15.149:3300/0 conn(0x7f390c15c760 0x7f390c0b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:45.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.235+0000 7f3911952640 1 --2- 172.21.15.87:0/4285371571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f390c15b490 0x7f390c15c220 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:45.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.235+0000 7f3911952640 1 -- 172.21.15.87:0/4285371571 >> 172.21.15.87:0/4285371571 conn(0x7f390c0d1110 msgr2=0x7f390c0d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:45.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.235+0000 7f3911952640 1 -- 172.21.15.87:0/4285371571 shutdown_connections 2024-09-17T03:14:45.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.235+0000 7f3911952640 1 -- 172.21.15.87:0/4285371571 wait complete. 2024-09-17T03:14:45.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.237+0000 7f3911952640 1 Processor -- start 2024-09-17T03:14:45.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.237+0000 7f3911952640 1 -- start start 2024-09-17T03:14:45.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.238+0000 7f3911952640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f390c0d04e0 0x7f390c0b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:45.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.238+0000 7f3911952640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f390c15b490 0x7f390c1f75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:45.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.238+0000 7f3911952640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f390c15c760 0x7f390c14af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:45.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.238+0000 7f3911952640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f390c0cc620 con 0x7f390c15c760 2024-09-17T03:14:45.239 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.238+0000 7f3911952640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f390c0cc4a0 con 0x7f390c0d04e0 2024-09-17T03:14:45.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.238+0000 7f3911952640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f390c0cc7a0 con 0x7f390c15b490 2024-09-17T03:14:45.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.238+0000 7f390b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f390c15c760 0x7f390c14af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:45.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.239+0000 7f390b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f390c15c760 0x7f390c14af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58450/0 (socket says 172.21.15.87:58450) 2024-09-17T03:14:45.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.239+0000 7f390b7fe640 1 -- 172.21.15.87:0/781844566 learned_addr learned my addr 172.21.15.87:0/781844566 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:45.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.239+0000 7f390affd640 1 --2- 172.21.15.87:0/781844566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f390c0d04e0 0x7f390c0b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:45.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.239+0000 7f390a7fc640 1 --2- 172.21.15.87:0/781844566 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f390c15b490 0x7f390c1f75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:45.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.239+0000 7f390b7fe640 1 -- 172.21.15.87:0/781844566 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f390c15b490 msgr2=0x7f390c1f75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:45.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.239+0000 7f390b7fe640 1 --2- 172.21.15.87:0/781844566 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f390c15b490 0x7f390c1f75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:45.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.239+0000 7f390b7fe640 1 -- 172.21.15.87:0/781844566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f390c0d04e0 msgr2=0x7f390c0b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:45.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.239+0000 7f390b7fe640 1 --2- 172.21.15.87:0/781844566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f390c0d04e0 0x7f390c0b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:45.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.239+0000 7f390b7fe640 1 -- 172.21.15.87:0/781844566 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390c0b5890 con 0x7f390c15c760 2024-09-17T03:14:45.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.239+0000 7f390b7fe640 1 --2- 172.21.15.87:0/781844566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f390c15c760 0x7f390c14af90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00b9e0 tx=0x7f38fc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:45.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.239+0000 7f38f3fff640 1 -- 172.21.15.87:0/781844566 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc019070 con 0x7f390c15c760 2024-09-17T03:14:45.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.239+0000 7f38f3fff640 1 -- 172.21.15.87:0/781844566 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38fc00cc30 con 0x7f390c15c760 2024-09-17T03:14:45.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.239+0000 7f3911952640 1 -- 172.21.15.87:0/781844566 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f390c0b5b70 con 0x7f390c15c760 2024-09-17T03:14:45.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.239+0000 7f38f3fff640 1 -- 172.21.15.87:0/781844566 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc0146a0 con 0x7f390c15c760 2024-09-17T03:14:45.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.239+0000 7f3911952640 1 -- 172.21.15.87:0/781844566 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f390c0b6130 con 0x7f390c15c760 2024-09-17T03:14:45.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.240+0000 7f3911952640 1 -- 172.21.15.87:0/781844566 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f390c0b5600 con 0x7f390c15c760 2024-09-17T03:14:45.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.241+0000 7f38f3fff640 1 -- 172.21.15.87:0/781844566 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f38fc01c070 con 0x7f390c15c760 2024-09-17T03:14:45.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.241+0000 7f38f3fff640 1 --2- 172.21.15.87:0/781844566 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f38f8078b90 0x7f38f807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:45.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.241+0000 7f38f3fff640 1 -- 172.21.15.87:0/781844566 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f38fc09a910 con 0x7f390c15c760 2024-09-17T03:14:45.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.242+0000 7f390affd640 1 --2- 172.21.15.87:0/781844566 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f38f8078b90 0x7f38f807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:45.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.242+0000 7f390affd640 1 --2- 172.21.15.87:0/781844566 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f38f8078b90 0x7f38f807b0a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3904006fd0 tx=0x7f3904009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:45.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.246+0000 7f38f3fff640 1 -- 172.21.15.87:0/781844566 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f38fc00ce20 con 0x7f390c15c760 2024-09-17T03:14:45.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.457+0000 7f3911952640 1 -- 172.21.15.87:0/781844566 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f390c0cccd0 con 0x7f390c15c760 2024-09-17T03:14:45.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.457+0000 7f38f3fff640 1 -- 172.21.15.87:0/781844566 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5217 (secure 0 0 0) 0x7f38fc062700 con 0x7f390c15c760 2024-09-17T03:14:45.458 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:45.458 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":6,"btime":"2024-09-17T03:14:16:330267+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4289,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6835/110106418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":110106418},{"type":"v1","addr":"172.21.15.149:6835","nonce":110106418}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1336626873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6836","nonce":1336626873},{"type":"v1","addr":"172.21.15.149:6837","nonce":1336626873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4395,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6833/666860906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":666860906},{"type":"v1","addr":"172.21.15.87:6833","nonce":666860906}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:14:13.672621+0000","modified":"2024-09-17T03:14:15.675568+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4398},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4398":{"gid":4398,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/1831073338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1831073338},{"type":"v1","addr":"172.21.15.87:6835","nonce":1831073338}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4398,"qdb_cluster":[4398]},"id":1}]} 2024-09-17T03:14:45.458 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 6 2024-09-17T03:14:45.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.460+0000 7f3911952640 1 -- 172.21.15.87:0/781844566 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f38f8078b90 msgr2=0x7f38f807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:45.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.460+0000 7f3911952640 1 --2- 172.21.15.87:0/781844566 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f38f8078b90 0x7f38f807b0a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3904006fd0 tx=0x7f3904009060 comp rx=0 tx=0).stop 2024-09-17T03:14:45.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.460+0000 7f3911952640 1 -- 172.21.15.87:0/781844566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f390c15c760 msgr2=0x7f390c14af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:45.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.460+0000 7f3911952640 1 --2- 172.21.15.87:0/781844566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f390c15c760 0x7f390c14af90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00b9e0 tx=0x7f38fc00bea0 comp rx=0 tx=0).stop 2024-09-17T03:14:45.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.460+0000 7f3911952640 1 -- 172.21.15.87:0/781844566 shutdown_connections 2024-09-17T03:14:45.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.460+0000 7f3911952640 1 --2- 172.21.15.87:0/781844566 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f38f8078b90 0x7f38f807b0a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:45.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.460+0000 7f3911952640 1 --2- 172.21.15.87:0/781844566 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f390c15c760 0x7f390c14af90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:45.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.460+0000 7f3911952640 1 --2- 172.21.15.87:0/781844566 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f390c15b490 0x7f390c1f75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:45.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.460+0000 7f3911952640 1 --2- 172.21.15.87:0/781844566 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f390c0d04e0 0x7f390c0b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:45.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.460+0000 7f3911952640 1 -- 172.21.15.87:0/781844566 >> 172.21.15.87:0/781844566 conn(0x7f390c0d1110 msgr2=0x7f390c070ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:45.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.460+0000 7f3911952640 1 -- 172.21.15.87:0/781844566 shutdown_connections 2024-09-17T03:14:45.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.460+0000 7f3911952640 1 -- 172.21.15.87:0/781844566 wait complete. 2024-09-17T03:14:45.471 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:14:45.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.585+0000 7fb808b82640 1 -- 172.21.15.87:0/811172166 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8040cf020 msgr2=0x7fb8040b5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:45.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.585+0000 7fb808b82640 1 --2- 172.21.15.87:0/811172166 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8040cf020 0x7fb8040b5ea0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8005fd0 tx=0x7fb7f802a5d0 comp rx=0 tx=0).stop 2024-09-17T03:14:45.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.585+0000 7fb808b82640 1 -- 172.21.15.87:0/811172166 shutdown_connections 2024-09-17T03:14:45.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.585+0000 7fb808b82640 1 --2- 172.21.15.87:0/811172166 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8040cf430 0x7fb8040b63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:45.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.585+0000 7fb808b82640 1 --2- 172.21.15.87:0/811172166 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8040cf020 0x7fb8040b5ea0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:45.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.585+0000 7fb808b82640 1 -- 172.21.15.87:0/811172166 >> 172.21.15.87:0/811172166 conn(0x7fb804069900 msgr2=0x7fb8040d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:45.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.585+0000 7fb808b82640 1 -- 172.21.15.87:0/811172166 shutdown_connections 2024-09-17T03:14:45.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.585+0000 7fb808b82640 1 -- 172.21.15.87:0/811172166 wait complete. 2024-09-17T03:14:45.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.587+0000 7fb808b82640 1 Processor -- start 2024-09-17T03:14:45.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.587+0000 7fb808b82640 1 -- start start 2024-09-17T03:14:45.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.588+0000 7fb808b82640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8040cf020 0x7fb804148080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:45.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.588+0000 7fb808b82640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8040cf430 0x7fb8041485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:45.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.588+0000 7fb808b82640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb80415c5f0 0x7fb804159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:45.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.588+0000 7fb808b82640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb8040d33a0 con 0x7fb80415c5f0 2024-09-17T03:14:45.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.588+0000 7fb808b82640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb8040d3220 con 0x7fb8040cf430 2024-09-17T03:14:45.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.588+0000 7fb808b82640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb8040d3520 con 0x7fb8040cf020 2024-09-17T03:14:45.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.588+0000 7fb802d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb80415c5f0 0x7fb804159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:45.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.588+0000 7fb802d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb80415c5f0 0x7fb804159510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33986/0 (socket says 172.21.15.87:33986) 2024-09-17T03:14:45.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.588+0000 7fb802d9d640 1 -- 172.21.15.87:0/622812696 learned_addr learned my addr 172.21.15.87:0/622812696 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:45.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.588+0000 7fb80259c640 1 --2- 172.21.15.87:0/622812696 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8040cf020 0x7fb804148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:45.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.588+0000 7fb801d9b640 1 --2- 172.21.15.87:0/622812696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8040cf430 0x7fb8041485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:45.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.589+0000 7fb802d9d640 1 -- 172.21.15.87:0/622812696 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8040cf020 msgr2=0x7fb804148080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:45.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.589+0000 7fb802d9d640 1 --2- 172.21.15.87:0/622812696 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8040cf020 0x7fb804148080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:45.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.589+0000 7fb802d9d640 1 -- 172.21.15.87:0/622812696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8040cf430 msgr2=0x7fb8041485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:45.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.589+0000 7fb802d9d640 1 --2- 172.21.15.87:0/622812696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8040cf430 0x7fb8041485c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:45.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.589+0000 7fb802d9d640 1 -- 172.21.15.87:0/622812696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8040cd290 con 0x7fb80415c5f0 2024-09-17T03:14:45.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.589+0000 7fb802d9d640 1 --2- 172.21.15.87:0/622812696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb80415c5f0 0x7fb804159510 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00e9c0 tx=0x7fb7fc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:45.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.589+0000 7fb7f2ffd640 1 -- 172.21.15.87:0/622812696 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc004d20 con 0x7fb80415c5f0 2024-09-17T03:14:45.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.589+0000 7fb7f2ffd640 1 -- 172.21.15.87:0/622812696 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7fc00cb20 con 0x7fb80415c5f0 2024-09-17T03:14:45.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.589+0000 7fb808b82640 1 -- 172.21.15.87:0/622812696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8040cd570 con 0x7fb80415c5f0 2024-09-17T03:14:45.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.589+0000 7fb7f2ffd640 1 -- 172.21.15.87:0/622812696 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc005020 con 0x7fb80415c5f0 2024-09-17T03:14:45.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.589+0000 7fb808b82640 1 -- 172.21.15.87:0/622812696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8040cdb30 con 0x7fb80415c5f0 2024-09-17T03:14:45.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.590+0000 7fb808b82640 1 -- 172.21.15.87:0/622812696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8040cd000 con 0x7fb80415c5f0 2024-09-17T03:14:45.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.591+0000 7fb7f2ffd640 1 -- 172.21.15.87:0/622812696 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb7fc0051c0 con 0x7fb80415c5f0 2024-09-17T03:14:45.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.591+0000 7fb7f2ffd640 1 --2- 172.21.15.87:0/622812696 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb7e4078b90 0x7fb7e407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:45.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.591+0000 7fb7f2ffd640 1 -- 172.21.15.87:0/622812696 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb7fc09b520 con 0x7fb80415c5f0 2024-09-17T03:14:45.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.591+0000 7fb80259c640 1 --2- 172.21.15.87:0/622812696 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb7e4078b90 0x7fb7e407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:45.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.592+0000 7fb80259c640 1 --2- 172.21.15.87:0/622812696 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb7e4078b90 0x7fb7e407b0a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4001720 tx=0x7fb7f4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:45.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.596+0000 7fb7f2ffd640 1 -- 172.21.15.87:0/622812696 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb7fc016030 con 0x7fb80415c5f0 2024-09-17T03:14:45.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.771+0000 7fb808b82640 1 -- 172.21.15.87:0/622812696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8040d3980 con 0x7fb80415c5f0 2024-09-17T03:14:45.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.772+0000 7fb7f2ffd640 1 -- 172.21.15.87:0/622812696 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16693 (secure 0 0 0) 0x7fb7fc0633a0 con 0x7fb80415c5f0 2024-09-17T03:14:45.772 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:45.772 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":23,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:14:15.323716+0000","last_up_change":"2024-09-17T03:14:01.833605+0000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:14:03.433404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:14:11.525547+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-17T03:14:12.626396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6825","nonce":2430409749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6827","nonce":2430409749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6831","nonce":2430409749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6829","nonce":2430409749}]},"public_addr":"172.21.15.87:6825/2430409749","cluster_addr":"172.21.15.87:6827/2430409749","heartbeat_back_addr":"172.21.15.87:6831/2430409749","heartbeat_front_addr":"172.21.15.87:6829/2430409749","state":["exists","up"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6809","nonce":3467605435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6811","nonce":3467605435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6815","nonce":3467605435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6813","nonce":3467605435}]},"public_addr":"172.21.15.87:6809/3467605435","cluster_addr":"172.21.15.87:6811/3467605435","heartbeat_back_addr":"172.21.15.87:6815/3467605435","heartbeat_front_addr":"172.21.15.87:6813/3467605435","state":["exists","up"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6801","nonce":1450834766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6803","nonce":1450834766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6807","nonce":1450834766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6805","nonce":1450834766}]},"public_addr":"172.21.15.87:6801/1450834766","cluster_addr":"172.21.15.87:6803/1450834766","heartbeat_back_addr":"172.21.15.87:6807/1450834766","heartbeat_front_addr":"172.21.15.87:6805/1450834766","state":["exists","up"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6817","nonce":4253161216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6819","nonce":4253161216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6823","nonce":4253161216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6821","nonce":4253161216}]},"public_addr":"172.21.15.87:6817/4253161216","cluster_addr":"172.21.15.87:6819/4253161216","heartbeat_back_addr":"172.21.15.87:6823/4253161216","heartbeat_front_addr":"172.21.15.87:6821/4253161216","state":["exists","up"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6803","nonce":2414907089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6805","nonce":2414907089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6809","nonce":2414907089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6807","nonce":2414907089}]},"public_addr":"172.21.15.149:6803/2414907089","cluster_addr":"172.21.15.149:6805/2414907089","heartbeat_back_addr":"172.21.15.149:6809/2414907089","heartbeat_front_addr":"172.21.15.149:6807/2414907089","state":["exists","up"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6827","nonce":2182999530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6829","nonce":2182999530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6833","nonce":2182999530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6831","nonce":2182999530}]},"public_addr":"172.21.15.149:6827/2182999530","cluster_addr":"172.21.15.149:6829/2182999530","heartbeat_back_addr":"172.21.15.149:6833/2182999530","heartbeat_front_addr":"172.21.15.149:6831/2182999530","state":["exists","up"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6811","nonce":4289519146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6813","nonce":4289519146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6817","nonce":4289519146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6815","nonce":4289519146}]},"public_addr":"172.21.15.149:6811/4289519146","cluster_addr":"172.21.15.149:6813/4289519146","heartbeat_back_addr":"172.21.15.149:6817/4289519146","heartbeat_front_addr":"172.21.15.149:6815/4289519146","state":["exists","up"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6819","nonce":2771010477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6821","nonce":2771010477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6825","nonce":2771010477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6823","nonce":2771010477}]},"public_addr":"172.21.15.149:6819/2771010477","cluster_addr":"172.21.15.149:6821/2771010477","heartbeat_back_addr":"172.21.15.149:6825/2771010477","heartbeat_front_addr":"172.21.15.149:6823/2771010477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.313297+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.642861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.654781+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.302606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.385791+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.915266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.121365+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.394432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:14:45.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.774+0000 7fb808b82640 1 -- 172.21.15.87:0/622812696 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb7e4078b90 msgr2=0x7fb7e407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:45.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.774+0000 7fb808b82640 1 --2- 172.21.15.87:0/622812696 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb7e4078b90 0x7fb7e407b0a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb7f4001720 tx=0x7fb7f4009040 comp rx=0 tx=0).stop 2024-09-17T03:14:45.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.774+0000 7fb808b82640 1 -- 172.21.15.87:0/622812696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb80415c5f0 msgr2=0x7fb804159510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:45.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.774+0000 7fb808b82640 1 --2- 172.21.15.87:0/622812696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb80415c5f0 0x7fb804159510 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc00e9c0 tx=0x7fb7fc00ee80 comp rx=0 tx=0).stop 2024-09-17T03:14:45.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.774+0000 7fb808b82640 1 -- 172.21.15.87:0/622812696 shutdown_connections 2024-09-17T03:14:45.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.774+0000 7fb808b82640 1 --2- 172.21.15.87:0/622812696 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb7e4078b90 0x7fb7e407b0a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:45.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.774+0000 7fb808b82640 1 --2- 172.21.15.87:0/622812696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb80415c5f0 0x7fb804159510 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:45.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.774+0000 7fb808b82640 1 --2- 172.21.15.87:0/622812696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8040cf430 0x7fb8041485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:45.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.774+0000 7fb808b82640 1 --2- 172.21.15.87:0/622812696 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb8040cf020 0x7fb804148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:45.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.774+0000 7fb808b82640 1 -- 172.21.15.87:0/622812696 >> 172.21.15.87:0/622812696 conn(0x7fb804069900 msgr2=0x7fb80406de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:45.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.774+0000 7fb808b82640 1 -- 172.21.15.87:0/622812696 shutdown_connections 2024-09-17T03:14:45.775 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.774+0000 7fb808b82640 1 -- 172.21.15.87:0/622812696 wait complete. 2024-09-17T03:14:45.785 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T03:14:45.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.898+0000 7fe98fe3f640 1 -- 172.21.15.87:0/2569500832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe988072530 msgr2=0x7fe9880b4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:45.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.898+0000 7fe98fe3f640 1 --2- 172.21.15.87:0/2569500832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe988072530 0x7fe9880b4820 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe980009830 tx=0x7fe98002a510 comp rx=0 tx=0).stop 2024-09-17T03:14:45.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.898+0000 7fe98fe3f640 1 -- 172.21.15.87:0/2569500832 shutdown_connections 2024-09-17T03:14:45.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.898+0000 7fe98fe3f640 1 --2- 172.21.15.87:0/2569500832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe988072530 0x7fe9880b4820 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:45.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.898+0000 7fe98fe3f640 1 -- 172.21.15.87:0/2569500832 >> 172.21.15.87:0/2569500832 conn(0x7fe9880d1180 msgr2=0x7fe9880d1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:45.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.898+0000 7fe98fe3f640 1 -- 172.21.15.87:0/2569500832 shutdown_connections 2024-09-17T03:14:45.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.898+0000 7fe98fe3f640 1 -- 172.21.15.87:0/2569500832 wait complete. 2024-09-17T03:14:45.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.899+0000 7fe98fe3f640 1 Processor -- start 2024-09-17T03:14:45.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.900+0000 7fe98fe3f640 1 -- start start 2024-09-17T03:14:45.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.901+0000 7fe98fe3f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe988072120 0x7fe9881504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:45.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.901+0000 7fe98fe3f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe988072530 0x7fe98806cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:45.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.901+0000 7fe98fe3f640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe988151170 0x7fe98806d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:45.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.901+0000 7fe98fe3f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe9880742c0 con 0x7fe988072530 2024-09-17T03:14:45.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.901+0000 7fe98fe3f640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe988074140 con 0x7fe988072120 2024-09-17T03:14:45.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.901+0000 7fe98fe3f640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe988074440 con 0x7fe988151170 2024-09-17T03:14:45.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.901+0000 7fe98d3da640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe988072530 0x7fe98806cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:45.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.901+0000 7fe98d3da640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe988072530 0x7fe98806cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34000/0 (socket says 172.21.15.87:34000) 2024-09-17T03:14:45.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.901+0000 7fe98d3da640 1 -- 172.21.15.87:0/3242988102 learned_addr learned my addr 172.21.15.87:0/3242988102 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:45.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.901+0000 7fe98dbdb640 1 --2- 172.21.15.87:0/3242988102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe988072120 0x7fe9881504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:45.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.901+0000 7fe98d3da640 1 -- 172.21.15.87:0/3242988102 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe988151170 msgr2=0x7fe98806d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:45.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.901+0000 7fe98e3dc640 1 --2- 172.21.15.87:0/3242988102 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe988151170 0x7fe98806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:45.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.901+0000 7fe98d3da640 1 --2- 172.21.15.87:0/3242988102 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe988151170 0x7fe98806d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:45.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.901+0000 7fe98d3da640 1 -- 172.21.15.87:0/3242988102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe988072120 msgr2=0x7fe9881504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:45.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.901+0000 7fe98d3da640 1 --2- 172.21.15.87:0/3242988102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe988072120 0x7fe9881504e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:45.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.901+0000 7fe98d3da640 1 -- 172.21.15.87:0/3242988102 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe988150a20 con 0x7fe988072530 2024-09-17T03:14:45.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.901+0000 7fe98d3da640 1 --2- 172.21.15.87:0/3242988102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe988072530 0x7fe98806cde0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe97800c3d0 tx=0x7fe97800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:45.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.901+0000 7fe97e7fc640 1 -- 172.21.15.87:0/3242988102 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe978006890 con 0x7fe988072530 2024-09-17T03:14:45.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.902+0000 7fe97e7fc640 1 -- 172.21.15.87:0/3242988102 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe978006b20 con 0x7fe988072530 2024-09-17T03:14:45.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.902+0000 7fe98fe3f640 1 -- 172.21.15.87:0/3242988102 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9881f6d40 con 0x7fe988072530 2024-09-17T03:14:45.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.902+0000 7fe97e7fc640 1 -- 172.21.15.87:0/3242988102 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe978007020 con 0x7fe988072530 2024-09-17T03:14:45.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.902+0000 7fe98fe3f640 1 -- 172.21.15.87:0/3242988102 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9881f7300 con 0x7fe988072530 2024-09-17T03:14:45.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.902+0000 7fe98fe3f640 1 -- 172.21.15.87:0/3242988102 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe988074930 con 0x7fe988072530 2024-09-17T03:14:45.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.903+0000 7fe97e7fc640 1 -- 172.21.15.87:0/3242988102 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe978019020 con 0x7fe988072530 2024-09-17T03:14:45.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.904+0000 7fe97e7fc640 1 --2- 172.21.15.87:0/3242988102 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe96c078bc0 0x7fe96c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:45.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.904+0000 7fe97e7fc640 1 -- 172.21.15.87:0/3242988102 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe97809a440 con 0x7fe988072530 2024-09-17T03:14:45.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.904+0000 7fe98dbdb640 1 --2- 172.21.15.87:0/3242988102 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe96c078bc0 0x7fe96c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:45.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.905+0000 7fe98dbdb640 1 --2- 172.21.15.87:0/3242988102 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe96c078bc0 0x7fe96c07b0d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe984007dc0 tx=0x7fe9840072f0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:45.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:45.908+0000 7fe97e7fc640 1 -- 172.21.15.87:0/3242988102 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe988150a20 con 0x7fe988072530 2024-09-17T03:14:46.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.118+0000 7fe98fe3f640 1 -- 172.21.15.87:0/3242988102 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe9880b09c0 con 0x7fe988072530 2024-09-17T03:14:46.118 INFO:tasks.ceph.mon.a.smithi087.stderr:2024-09-17T03:14:46.119+0000 7fafd01f9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T03:14:46.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.120+0000 7fe97e7fc640 1 -- 172.21.15.87:0/3242988102 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) ==== 161+0+0 (secure 0 0 0) 0x7fe978062880 con 0x7fe988072530 2024-09-17T03:14:46.120 INFO:teuthology.orchestra.run.smithi087.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T03:14:46.123 INFO:tasks.ceph.mds.a.smithi087.stderr: -3523> 2024-09-17T03:14:08.275+0000 7f5cee049640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6833/0: (98) Address already in use 2024-09-17T03:14:46.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.123+0000 7fe98fe3f640 1 -- 172.21.15.87:0/3242988102 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe96c078bc0 msgr2=0x7fe96c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:46.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.123+0000 7fe98fe3f640 1 --2- 172.21.15.87:0/3242988102 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe96c078bc0 0x7fe96c07b0d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe984007dc0 tx=0x7fe9840072f0 comp rx=0 tx=0).stop 2024-09-17T03:14:46.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.123+0000 7fe98fe3f640 1 -- 172.21.15.87:0/3242988102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe988072530 msgr2=0x7fe98806cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:46.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.123+0000 7fe98fe3f640 1 --2- 172.21.15.87:0/3242988102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe988072530 0x7fe98806cde0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe97800c3d0 tx=0x7fe97800c890 comp rx=0 tx=0).stop 2024-09-17T03:14:46.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.123+0000 7fe98fe3f640 1 -- 172.21.15.87:0/3242988102 shutdown_connections 2024-09-17T03:14:46.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.123+0000 7fe98fe3f640 1 --2- 172.21.15.87:0/3242988102 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe96c078bc0 0x7fe96c07b0d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:46.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.123+0000 7fe98fe3f640 1 --2- 172.21.15.87:0/3242988102 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe988151170 0x7fe98806d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:46.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.123+0000 7fe98fe3f640 1 --2- 172.21.15.87:0/3242988102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe988072530 0x7fe98806cde0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:46.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.123+0000 7fe98fe3f640 1 --2- 172.21.15.87:0/3242988102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe988072120 0x7fe9881504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:46.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.123+0000 7fe98fe3f640 1 -- 172.21.15.87:0/3242988102 >> 172.21.15.87:0/3242988102 conn(0x7fe9880d1180 msgr2=0x7fe98806fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:46.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.123+0000 7fe98fe3f640 1 -- 172.21.15.87:0/3242988102 shutdown_connections 2024-09-17T03:14:46.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.123+0000 7fe98fe3f640 1 -- 172.21.15.87:0/3242988102 wait complete. 2024-09-17T03:14:46.125 INFO:tasks.ceph.mds.a.smithi087.stderr: -2136> 2024-09-17T03:14:19.290+0000 7f5ce803d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T03:14:46.135 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T03:14:46.177 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-09-17T03:14:46.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.252+0000 7f98aca0e640 1 -- 172.21.15.87:0/3341974354 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98a80b7d70 msgr2=0x7f98a815a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:46.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.252+0000 7f98aca0e640 1 --2- 172.21.15.87:0/3341974354 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98a80b7d70 0x7f98a815a100 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f98a815a640 tx=0x7f98a002a980 comp rx=0 tx=0).stop 2024-09-17T03:14:46.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.252+0000 7f98aca0e640 1 -- 172.21.15.87:0/3341974354 shutdown_connections 2024-09-17T03:14:46.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.252+0000 7f98aca0e640 1 --2- 172.21.15.87:0/3341974354 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98a815d830 0x7f98a80b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:46.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.252+0000 7f98aca0e640 1 --2- 172.21.15.87:0/3341974354 >> v2:172.21.15.149:3300/0 conn(0x7f98a8160100 0x7f98a815d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:46.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.252+0000 7f98aca0e640 1 --2- 172.21.15.87:0/3341974354 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98a80b7d70 0x7f98a815a100 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:46.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.252+0000 7f98aca0e640 1 -- 172.21.15.87:0/3341974354 >> 172.21.15.87:0/3341974354 conn(0x7f98a8069330 msgr2=0x7f98a80ced20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:46.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.252+0000 7f98aca0e640 1 -- 172.21.15.87:0/3341974354 shutdown_connections 2024-09-17T03:14:46.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.252+0000 7f98aca0e640 1 -- 172.21.15.87:0/3341974354 wait complete. 2024-09-17T03:14:46.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.254+0000 7f98aca0e640 1 Processor -- start 2024-09-17T03:14:46.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.254+0000 7f98aca0e640 1 -- start start 2024-09-17T03:14:46.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.255+0000 7f98aca0e640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98a80b7d70 0x7f98a81595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:46.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.255+0000 7f98aca0e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98a815d830 0x7f98a80b0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:46.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.255+0000 7f98aca0e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98a8160100 0x7f98a80c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:46.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.255+0000 7f98aca0e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f98a80cfff0 con 0x7f98a815d830 2024-09-17T03:14:46.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.255+0000 7f98aca0e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f98a80cfe70 con 0x7f98a8160100 2024-09-17T03:14:46.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.255+0000 7f98aca0e640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f98a80d0170 con 0x7f98a80b7d70 2024-09-17T03:14:46.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.256+0000 7f98a5d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98a815d830 0x7f98a80b0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:46.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.256+0000 7f98a5d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98a815d830 0x7f98a80b0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34042/0 (socket says 172.21.15.87:34042) 2024-09-17T03:14:46.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.256+0000 7f98a5d9b640 1 -- 172.21.15.87:0/1973058450 learned_addr learned my addr 172.21.15.87:0/1973058450 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:46.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.256+0000 7f98a659c640 1 --2- 172.21.15.87:0/1973058450 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98a80b7d70 0x7f98a81595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:46.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.256+0000 7f98a6d9d640 1 --2- 172.21.15.87:0/1973058450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98a8160100 0x7f98a80c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:46.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.256+0000 7f98a5d9b640 1 -- 172.21.15.87:0/1973058450 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98a80b7d70 msgr2=0x7f98a81595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:46.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.256+0000 7f98a5d9b640 1 --2- 172.21.15.87:0/1973058450 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98a80b7d70 0x7f98a81595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:46.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.256+0000 7f98a5d9b640 1 -- 172.21.15.87:0/1973058450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98a8160100 msgr2=0x7f98a80c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:46.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.256+0000 7f98a5d9b640 1 --2- 172.21.15.87:0/1973058450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98a8160100 0x7f98a80c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:46.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.256+0000 7f98a5d9b640 1 -- 172.21.15.87:0/1973058450 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a8159b30 con 0x7f98a815d830 2024-09-17T03:14:46.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.256+0000 7f98a659c640 1 --2- 172.21.15.87:0/1973058450 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98a80b7d70 0x7f98a81595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:46.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.256+0000 7f98a6d9d640 1 --2- 172.21.15.87:0/1973058450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98a8160100 0x7f98a80c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:46.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.256+0000 7f98a5d9b640 1 --2- 172.21.15.87:0/1973058450 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98a815d830 0x7f98a80b0e80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f989000d6d0 tx=0x7f989000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:46.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.256+0000 7f989effd640 1 -- 172.21.15.87:0/1973058450 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890017070 con 0x7f98a815d830 2024-09-17T03:14:46.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.256+0000 7f989effd640 1 -- 172.21.15.87:0/1973058450 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9890009b10 con 0x7f98a815d830 2024-09-17T03:14:46.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.256+0000 7f98aca0e640 1 -- 172.21.15.87:0/1973058450 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a80b13c0 con 0x7f98a815d830 2024-09-17T03:14:46.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.256+0000 7f989effd640 1 -- 172.21.15.87:0/1973058450 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9890010460 con 0x7f98a815d830 2024-09-17T03:14:46.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.256+0000 7f98aca0e640 1 -- 172.21.15.87:0/1973058450 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a80c9310 con 0x7f98a815d830 2024-09-17T03:14:46.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.257+0000 7f98aca0e640 1 -- 172.21.15.87:0/1973058450 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a80cbdc0 con 0x7f98a815d830 2024-09-17T03:14:46.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.258+0000 7f989effd640 1 -- 172.21.15.87:0/1973058450 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f989001a070 con 0x7f98a815d830 2024-09-17T03:14:46.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.258+0000 7f989effd640 1 --2- 172.21.15.87:0/1973058450 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9888078b90 0x7f988807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:46.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.258+0000 7f989effd640 1 -- 172.21.15.87:0/1973058450 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5795+0+0 (secure 0 0 0) 0x7f989009b0c0 con 0x7f98a815d830 2024-09-17T03:14:46.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.259+0000 7f98a659c640 1 --2- 172.21.15.87:0/1973058450 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9888078b90 0x7f988807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:46.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.259+0000 7f98a659c640 1 --2- 172.21.15.87:0/1973058450 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9888078b90 0x7f988807b0a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f98a0035000 tx=0x7f98a0004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:46.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.263+0000 7f989effd640 1 -- 172.21.15.87:0/1973058450 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9890063530 con 0x7f98a815d830 2024-09-17T03:14:46.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:46.479+0000 7f98aca0e640 1 -- 172.21.15.87:0/1973058450 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f98a80d0700 con 0x7f98a815d830 2024-09-17T03:14:47.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.122+0000 7f989effd640 1 -- 172.21.15.87:0/1973058450 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) ==== 104+0+0 (secure 0 0 0) 0x7f9890062e30 con 0x7f98a815d830 2024-09-17T03:14:47.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.124+0000 7f98aca0e640 1 -- 172.21.15.87:0/1973058450 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9888078b90 msgr2=0x7f988807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:47.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.124+0000 7f98aca0e640 1 --2- 172.21.15.87:0/1973058450 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9888078b90 0x7f988807b0a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f98a0035000 tx=0x7f98a0004ab0 comp rx=0 tx=0).stop 2024-09-17T03:14:47.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.124+0000 7f98aca0e640 1 -- 172.21.15.87:0/1973058450 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98a815d830 msgr2=0x7f98a80b0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:47.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.124+0000 7f98aca0e640 1 --2- 172.21.15.87:0/1973058450 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98a815d830 0x7f98a80b0e80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f989000d6d0 tx=0x7f989000db90 comp rx=0 tx=0).stop 2024-09-17T03:14:47.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.124+0000 7f98aca0e640 1 -- 172.21.15.87:0/1973058450 shutdown_connections 2024-09-17T03:14:47.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.124+0000 7f98aca0e640 1 --2- 172.21.15.87:0/1973058450 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9888078b90 0x7f988807b0a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:47.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.124+0000 7f98aca0e640 1 --2- 172.21.15.87:0/1973058450 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98a8160100 0x7f98a80c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:47.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.124+0000 7f98aca0e640 1 --2- 172.21.15.87:0/1973058450 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98a815d830 0x7f98a80b0e80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:47.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.124+0000 7f98aca0e640 1 --2- 172.21.15.87:0/1973058450 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98a80b7d70 0x7f98a81595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:47.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.124+0000 7f98aca0e640 1 -- 172.21.15.87:0/1973058450 >> 172.21.15.87:0/1973058450 conn(0x7f98a8069330 msgr2=0x7f98a815e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:47.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.125+0000 7f98aca0e640 1 -- 172.21.15.87:0/1973058450 shutdown_connections 2024-09-17T03:14:47.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.125+0000 7f98aca0e640 1 -- 172.21.15.87:0/1973058450 wait complete. 2024-09-17T03:14:47.138 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T03:14:47.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.298+0000 7fee7c38a640 1 -- 172.21.15.87:0/1069683494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee7415d870 msgr2=0x7fee740ccf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:47.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.298+0000 7fee7c38a640 1 --2- 172.21.15.87:0/1069683494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee7415d870 0x7fee740ccf80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fee70009830 tx=0x7fee7002a500 comp rx=0 tx=0).stop 2024-09-17T03:14:47.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.298+0000 7fee7c38a640 1 -- 172.21.15.87:0/1069683494 shutdown_connections 2024-09-17T03:14:47.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.298+0000 7fee7c38a640 1 --2- 172.21.15.87:0/1069683494 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fee74160170 0x7fee74160580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:47.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.298+0000 7fee7c38a640 1 --2- 172.21.15.87:0/1069683494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee7415d870 0x7fee740ccf80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:47.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.298+0000 7fee7c38a640 1 -- 172.21.15.87:0/1069683494 >> 172.21.15.87:0/1069683494 conn(0x7fee74069120 msgr2=0x7fee74069560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:47.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.298+0000 7fee7c38a640 1 -- 172.21.15.87:0/1069683494 shutdown_connections 2024-09-17T03:14:47.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.298+0000 7fee7c38a640 1 -- 172.21.15.87:0/1069683494 wait complete. 2024-09-17T03:14:47.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.300+0000 7fee7c38a640 1 Processor -- start 2024-09-17T03:14:47.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.300+0000 7fee7c38a640 1 -- start start 2024-09-17T03:14:47.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.301+0000 7fee7c38a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee740d2210 0x7fee740c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:47.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.301+0000 7fee7c38a640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fee7415d870 0x7fee740b3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:47.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.301+0000 7fee7c38a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee74160170 0x7fee740c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:47.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.301+0000 7fee7c38a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fee74153210 con 0x7fee74160170 2024-09-17T03:14:47.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.301+0000 7fee7c38a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fee74153090 con 0x7fee740d2210 2024-09-17T03:14:47.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.301+0000 7fee7c38a640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fee74153390 con 0x7fee7415d870 2024-09-17T03:14:47.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.301+0000 7fee7a927640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee74160170 0x7fee740c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:47.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.302+0000 7fee7a927640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee74160170 0x7fee740c7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34054/0 (socket says 172.21.15.87:34054) 2024-09-17T03:14:47.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.302+0000 7fee7a927640 1 -- 172.21.15.87:0/3333118784 learned_addr learned my addr 172.21.15.87:0/3333118784 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:47.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.302+0000 7fee79925640 1 --2- 172.21.15.87:0/3333118784 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fee7415d870 0x7fee740b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:47.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.302+0000 7fee7a126640 1 --2- 172.21.15.87:0/3333118784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee740d2210 0x7fee740c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:47.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.302+0000 7fee7a927640 1 -- 172.21.15.87:0/3333118784 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fee7415d870 msgr2=0x7fee740b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:47.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.302+0000 7fee7a927640 1 --2- 172.21.15.87:0/3333118784 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fee7415d870 0x7fee740b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:47.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.302+0000 7fee7a927640 1 -- 172.21.15.87:0/3333118784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee740d2210 msgr2=0x7fee740c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:47.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.302+0000 7fee7a927640 1 --2- 172.21.15.87:0/3333118784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee740d2210 0x7fee740c9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:47.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.302+0000 7fee7a927640 1 -- 172.21.15.87:0/3333118784 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee7406f2e0 con 0x7fee74160170 2024-09-17T03:14:47.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.302+0000 7fee79925640 1 --2- 172.21.15.87:0/3333118784 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fee7415d870 0x7fee740b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:14:47.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.302+0000 7fee7a927640 1 --2- 172.21.15.87:0/3333118784 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee74160170 0x7fee740c7f80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00cab0 tx=0x7fee6c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:47.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.302+0000 7fee6affd640 1 -- 172.21.15.87:0/3333118784 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c007df0 con 0x7fee74160170 2024-09-17T03:14:47.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.302+0000 7fee6affd640 1 -- 172.21.15.87:0/3333118784 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee6c003a80 con 0x7fee74160170 2024-09-17T03:14:47.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.302+0000 7fee7c38a640 1 -- 172.21.15.87:0/3333118784 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee740b2860 con 0x7fee74160170 2024-09-17T03:14:47.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.302+0000 7fee6affd640 1 -- 172.21.15.87:0/3333118784 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee6c004020 con 0x7fee74160170 2024-09-17T03:14:47.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.302+0000 7fee7c38a640 1 -- 172.21.15.87:0/3333118784 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee7415f670 con 0x7fee74160170 2024-09-17T03:14:47.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.303+0000 7fee7c38a640 1 -- 172.21.15.87:0/3333118784 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee74153840 con 0x7fee74160170 2024-09-17T03:14:47.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.304+0000 7fee6affd640 1 -- 172.21.15.87:0/3333118784 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fee6c0041c0 con 0x7fee74160170 2024-09-17T03:14:47.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.304+0000 7fee6affd640 1 --2- 172.21.15.87:0/3333118784 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fee58078bc0 0x7fee5807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:47.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.304+0000 7fee6affd640 1 -- 172.21.15.87:0/3333118784 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5795+0+0 (secure 0 0 0) 0x7fee6c09b3d0 con 0x7fee74160170 2024-09-17T03:14:47.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.305+0000 7fee7a126640 1 --2- 172.21.15.87:0/3333118784 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fee58078bc0 0x7fee5807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:47.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.305+0000 7fee7a126640 1 --2- 172.21.15.87:0/3333118784 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fee58078bc0 0x7fee5807b0d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fee70034000 tx=0x7fee70004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:47.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.309+0000 7fee6affd640 1 -- 172.21.15.87:0/3333118784 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fee6c014070 con 0x7fee74160170 2024-09-17T03:14:47.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:47.485+0000 7fee7c38a640 1 -- 172.21.15.87:0/3333118784 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fee7415fca0 con 0x7fee74160170 2024-09-17T03:14:48.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.126+0000 7fee6affd640 1 -- 172.21.15.87:0/3333118784 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) ==== 181+0+0 (secure 0 0 0) 0x7fee6c0638d0 con 0x7fee74160170 2024-09-17T03:14:48.127 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' removed 2024-09-17T03:14:48.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.130+0000 7fee7c38a640 1 -- 172.21.15.87:0/3333118784 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fee58078bc0 msgr2=0x7fee5807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:48.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.130+0000 7fee7c38a640 1 --2- 172.21.15.87:0/3333118784 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fee58078bc0 0x7fee5807b0d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fee70034000 tx=0x7fee70004730 comp rx=0 tx=0).stop 2024-09-17T03:14:48.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.130+0000 7fee7c38a640 1 -- 172.21.15.87:0/3333118784 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee74160170 msgr2=0x7fee740c7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:48.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.130+0000 7fee7c38a640 1 --2- 172.21.15.87:0/3333118784 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee74160170 0x7fee740c7f80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fee6c00cab0 tx=0x7fee6c00cf70 comp rx=0 tx=0).stop 2024-09-17T03:14:48.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.130+0000 7fee7c38a640 1 -- 172.21.15.87:0/3333118784 shutdown_connections 2024-09-17T03:14:48.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.130+0000 7fee7c38a640 1 --2- 172.21.15.87:0/3333118784 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fee58078bc0 0x7fee5807b0d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:48.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.130+0000 7fee7c38a640 1 --2- 172.21.15.87:0/3333118784 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee74160170 0x7fee740c7f80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:48.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.130+0000 7fee7c38a640 1 --2- 172.21.15.87:0/3333118784 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fee7415d870 0x7fee740b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:48.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.130+0000 7fee7c38a640 1 --2- 172.21.15.87:0/3333118784 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fee740d2210 0x7fee740c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:48.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.130+0000 7fee7c38a640 1 -- 172.21.15.87:0/3333118784 >> 172.21.15.87:0/3333118784 conn(0x7fee74069120 msgr2=0x7fee7415ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:48.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.130+0000 7fee7c38a640 1 -- 172.21.15.87:0/3333118784 shutdown_connections 2024-09-17T03:14:48.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.130+0000 7fee7c38a640 1 -- 172.21.15.87:0/3333118784 wait complete. 2024-09-17T03:14:48.143 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T03:14:48.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.261+0000 7f6919894640 1 -- 172.21.15.87:0/3382042840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6900003fd0 msgr2=0x7f6900004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:48.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.261+0000 7f6919894640 1 --2- 172.21.15.87:0/3382042840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6900003fd0 0x7f6900004800 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f68fc009b50 tx=0x7f6900005830 comp rx=0 tx=0).stop 2024-09-17T03:14:48.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.261+0000 7f69137fe640 1 -- 172.21.15.87:0/3382042840 reap_dead start 2024-09-17T03:14:48.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.261+0000 7f6919894640 1 -- 172.21.15.87:0/3382042840 shutdown_connections 2024-09-17T03:14:48.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.261+0000 7f6919894640 1 -- 172.21.15.87:0/3382042840 >> 172.21.15.87:0/3382042840 conn(0x7f691409d7e0 msgr2=0x7f691408bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:48.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.261+0000 7f6919894640 1 -- 172.21.15.87:0/3382042840 shutdown_connections 2024-09-17T03:14:48.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.261+0000 7f6919894640 1 -- 172.21.15.87:0/3382042840 wait complete. 2024-09-17T03:14:48.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.263+0000 7f6919894640 1 Processor -- start 2024-09-17T03:14:48.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.263+0000 7f6919894640 1 -- start start 2024-09-17T03:14:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.264+0000 7f6919894640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6900003fd0 0x7f691415b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.264+0000 7f6919894640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69140b1040 0x7f691415d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.264+0000 7f6919894640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69140cac90 0x7f691415d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.264+0000 7f6919894640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6914161f50 con 0x7f69140b1040 2024-09-17T03:14:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.264+0000 7f6919894640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6914161dd0 con 0x7f6900003fd0 2024-09-17T03:14:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.264+0000 7f6919894640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f69141620d0 con 0x7f69140cac90 2024-09-17T03:14:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.264+0000 7f690a7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69140b1040 0x7f691415d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.264+0000 7f690a7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69140b1040 0x7f691415d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34068/0 (socket says 172.21.15.87:34068) 2024-09-17T03:14:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.264+0000 7f690a7fc640 1 -- 172.21.15.87:0/1564101125 learned_addr learned my addr 172.21.15.87:0/1564101125 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.265+0000 7f6912ffd640 1 --2- 172.21.15.87:0/1564101125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6900003fd0 0x7f691415b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.265+0000 7f69137fe640 1 --2- 172.21.15.87:0/1564101125 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69140cac90 0x7f691415d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.265+0000 7f690a7fc640 1 -- 172.21.15.87:0/1564101125 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69140cac90 msgr2=0x7f691415d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.265+0000 7f690a7fc640 1 --2- 172.21.15.87:0/1564101125 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69140cac90 0x7f691415d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.265+0000 7f690a7fc640 1 -- 172.21.15.87:0/1564101125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6900003fd0 msgr2=0x7f691415b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.265+0000 7f690a7fc640 1 --2- 172.21.15.87:0/1564101125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6900003fd0 0x7f691415b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.265+0000 7f690a7fc640 1 -- 172.21.15.87:0/1564101125 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69140b2df0 con 0x7f69140b1040 2024-09-17T03:14:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.265+0000 7f690a7fc640 1 --2- 172.21.15.87:0/1564101125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69140b1040 0x7f691415d2a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f68fc036040 tx=0x7f68fc033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.265+0000 7f6910ff9640 1 -- 172.21.15.87:0/1564101125 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc038040 con 0x7f69140b1040 2024-09-17T03:14:48.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.265+0000 7f6910ff9640 1 -- 172.21.15.87:0/1564101125 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68fc041070 con 0x7f69140b1040 2024-09-17T03:14:48.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.265+0000 7f6919894640 1 -- 172.21.15.87:0/1564101125 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69141eae10 con 0x7f69140b1040 2024-09-17T03:14:48.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.265+0000 7f6910ff9640 1 -- 172.21.15.87:0/1564101125 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68fc033d70 con 0x7f69140b1040 2024-09-17T03:14:48.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.265+0000 7f6919894640 1 -- 172.21.15.87:0/1564101125 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69140b1480 con 0x7f69140b1040 2024-09-17T03:14:48.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.266+0000 7f6919894640 1 -- 172.21.15.87:0/1564101125 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69140d1110 con 0x7f69140b1040 2024-09-17T03:14:48.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.267+0000 7f6910ff9640 1 -- 172.21.15.87:0/1564101125 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f68fc02b8a0 con 0x7f69140b1040 2024-09-17T03:14:48.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.267+0000 7f6910ff9640 1 --2- 172.21.15.87:0/1564101125 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f68f4078bc0 0x7f68f407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:48.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.267+0000 7f6910ff9640 1 -- 172.21.15.87:0/1564101125 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5357+0+0 (secure 0 0 0) 0x7f68fc0bb440 con 0x7f69140b1040 2024-09-17T03:14:48.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.267+0000 7f6912ffd640 1 --2- 172.21.15.87:0/1564101125 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f68f4078bc0 0x7f68f407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:48.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.268+0000 7f6912ffd640 1 --2- 172.21.15.87:0/1564101125 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f68f4078bc0 0x7f68f407b0d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6904000fd0 tx=0x7f6904006c50 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:48.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.272+0000 7f6910ff9640 1 -- 172.21.15.87:0/1564101125 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f68fc083b00 con 0x7f69140b1040 2024-09-17T03:14:48.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:48.445+0000 7f6919894640 1 -- 172.21.15.87:0/1564101125 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f69141f8ac0 con 0x7f69140b1040 2024-09-17T03:14:49.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.128+0000 7f6910ff9640 1 -- 172.21.15.87:0/1564101125 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) ==== 169+0+0 (secure 0 0 0) 0x7f68fc083400 con 0x7f69140b1040 2024-09-17T03:14:49.129 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' removed 2024-09-17T03:14:49.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.131+0000 7f6919894640 1 -- 172.21.15.87:0/1564101125 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f68f4078bc0 msgr2=0x7f68f407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:49.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.131+0000 7f6919894640 1 --2- 172.21.15.87:0/1564101125 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f68f4078bc0 0x7f68f407b0d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6904000fd0 tx=0x7f6904006c50 comp rx=0 tx=0).stop 2024-09-17T03:14:49.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.131+0000 7f6919894640 1 -- 172.21.15.87:0/1564101125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69140b1040 msgr2=0x7f691415d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:49.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.131+0000 7f6919894640 1 --2- 172.21.15.87:0/1564101125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69140b1040 0x7f691415d2a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f68fc036040 tx=0x7f68fc033670 comp rx=0 tx=0).stop 2024-09-17T03:14:49.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.131+0000 7f6919894640 1 -- 172.21.15.87:0/1564101125 shutdown_connections 2024-09-17T03:14:49.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.131+0000 7f6919894640 1 --2- 172.21.15.87:0/1564101125 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f68f4078bc0 0x7f68f407b0d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:49.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.131+0000 7f6919894640 1 --2- 172.21.15.87:0/1564101125 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f69140cac90 0x7f691415d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:49.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.131+0000 7f6919894640 1 --2- 172.21.15.87:0/1564101125 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69140b1040 0x7f691415d2a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:49.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.131+0000 7f6919894640 1 --2- 172.21.15.87:0/1564101125 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6900003fd0 0x7f691415b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:49.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.131+0000 7f6919894640 1 -- 172.21.15.87:0/1564101125 >> 172.21.15.87:0/1564101125 conn(0x7f691409d7e0 msgr2=0x7f69140d3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:49.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.131+0000 7f6919894640 1 -- 172.21.15.87:0/1564101125 shutdown_connections 2024-09-17T03:14:49.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.131+0000 7f6919894640 1 -- 172.21.15.87:0/1564101125 wait complete. 2024-09-17T03:14:49.148 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T03:14:49.148 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T03:14:49.148 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T03:14:49.148 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:14:49.148 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T03:14:49.149 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T03:14:49.149 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T03:14:49.149 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:14:49.149 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T03:14:49.149 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T03:14:49.149 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T03:14:49.150 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:14:49.150 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T03:14:49.150 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T03:14:49.150 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T03:14:49.150 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:14:49.159 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T03:14:49.160 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T03:14:49.161 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T03:14:49.161 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T03:14:49.162 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T03:14:49.162 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T03:14:49.163 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T03:14:49.163 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T03:14:49.204 INFO:tasks.ceph.mds.b:Started 2024-09-17T03:14:49.204 DEBUG:teuthology.parallel:result is None 2024-09-17T03:14:49.205 INFO:tasks.ceph.mds.d:Started 2024-09-17T03:14:49.205 DEBUG:teuthology.parallel:result is None 2024-09-17T03:14:49.205 INFO:tasks.ceph.mds.c:Started 2024-09-17T03:14:49.206 DEBUG:teuthology.parallel:result is None 2024-09-17T03:14:49.207 INFO:tasks.ceph.mds.a:Started 2024-09-17T03:14:49.207 DEBUG:teuthology.parallel:result is None 2024-09-17T03:14:49.208 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T03:14:49.295 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-09-17T03:14:49.296 INFO:tasks.ceph.mds.c.smithi087.stdout:starting mds.c at 2024-09-17T03:14:49.296 INFO:tasks.ceph.mds.c.smithi087.stderr:2024-09-17T03:14:49.296+0000 7f18d6332640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-09-17T03:14:49.298 INFO:tasks.ceph.mds.b.smithi149.stdout:starting mds.b at 2024-09-17T03:14:49.298 INFO:tasks.ceph.mds.d.smithi149.stdout:starting mds.d at 2024-09-17T03:14:49.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.328+0000 7f7bb6c27640 1 -- 172.21.15.87:0/3377356193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bb0071ed0 msgr2=0x7f7bb00b0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:49.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.328+0000 7f7bb6c27640 1 --2- 172.21.15.87:0/3377356193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bb0071ed0 0x7f7bb00b0710 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7bb01596d0 tx=0x7f7ba002a5f0 comp rx=0 tx=0).stop 2024-09-17T03:14:49.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.328+0000 7f7bb6c27640 1 -- 172.21.15.87:0/3377356193 shutdown_connections 2024-09-17T03:14:49.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.328+0000 7f7bb6c27640 1 --2- 172.21.15.87:0/3377356193 >> v2:172.21.15.149:3300/0 conn(0x7f7bb015f890 0x7f7bb015bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:49.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.328+0000 7f7bb6c27640 1 --2- 172.21.15.87:0/3377356193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bb0071ed0 0x7f7bb00b0710 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:49.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.328+0000 7f7bb6c27640 1 -- 172.21.15.87:0/3377356193 >> 172.21.15.87:0/3377356193 conn(0x7f7bb00b2d10 msgr2=0x7f7bb00b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:49.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.328+0000 7f7bb6c27640 1 -- 172.21.15.87:0/3377356193 shutdown_connections 2024-09-17T03:14:49.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.328+0000 7f7bb6c27640 1 -- 172.21.15.87:0/3377356193 wait complete. 2024-09-17T03:14:49.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.330+0000 7f7bb6c27640 1 Processor -- start 2024-09-17T03:14:49.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.330+0000 7f7bb6c27640 1 -- start start 2024-09-17T03:14:49.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.332+0000 7f7bb6c27640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7bb0071ed0 0x7f7bb0147880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:49.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.332+0000 7f7bb6c27640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bb0155ab0 0x7f7bb00b0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:49.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.332+0000 7f7bb6c27640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7bb01418a0 0x7f7bb00b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:49.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.332+0000 7f7bb6c27640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb015c250 con 0x7f7bb0155ab0 2024-09-17T03:14:49.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.332+0000 7f7bb6c27640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb015c0d0 con 0x7f7bb01418a0 2024-09-17T03:14:49.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.332+0000 7f7bb6c27640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f7bb015c3d0 con 0x7f7bb0071ed0 2024-09-17T03:14:49.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.332+0000 7f7baffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bb0155ab0 0x7f7bb00b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:49.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.332+0000 7f7baffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bb0155ab0 0x7f7bb00b0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34118/0 (socket says 172.21.15.87:34118) 2024-09-17T03:14:49.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.332+0000 7f7bb49c3640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7bb0071ed0 0x7f7bb0147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:49.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.332+0000 7f7baffff640 1 -- 172.21.15.87:0/2865415882 learned_addr learned my addr 172.21.15.87:0/2865415882 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:49.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.332+0000 7f7bb51c4640 1 --2- 172.21.15.87:0/2865415882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7bb01418a0 0x7f7bb00b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:49.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.332+0000 7f7baffff640 1 -- 172.21.15.87:0/2865415882 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7bb0071ed0 msgr2=0x7f7bb0147880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:49.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.332+0000 7f7baffff640 1 --2- 172.21.15.87:0/2865415882 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7bb0071ed0 0x7f7bb0147880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:49.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.332+0000 7f7baffff640 1 -- 172.21.15.87:0/2865415882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7bb01418a0 msgr2=0x7f7bb00b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:49.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.332+0000 7f7baffff640 1 --2- 172.21.15.87:0/2865415882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7bb01418a0 0x7f7bb00b0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:49.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.332+0000 7f7baffff640 1 -- 172.21.15.87:0/2865415882 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb0147dc0 con 0x7f7bb0155ab0 2024-09-17T03:14:49.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.332+0000 7f7baffff640 1 --2- 172.21.15.87:0/2865415882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bb0155ab0 0x7f7bb00b0710 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400c7b0 tx=0x7f7ba400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:49.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.333+0000 7f7bad7fa640 1 -- 172.21.15.87:0/2865415882 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4003b70 con 0x7f7bb0155ab0 2024-09-17T03:14:49.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.333+0000 7f7bad7fa640 1 -- 172.21.15.87:0/2865415882 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ba4003e00 con 0x7f7bb0155ab0 2024-09-17T03:14:49.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.333+0000 7f7bb6c27640 1 -- 172.21.15.87:0/2865415882 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb01459a0 con 0x7f7bb0155ab0 2024-09-17T03:14:49.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.333+0000 7f7bad7fa640 1 -- 172.21.15.87:0/2865415882 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba400aab0 con 0x7f7bb0155ab0 2024-09-17T03:14:49.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.333+0000 7f7bb6c27640 1 -- 172.21.15.87:0/2865415882 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb0145c40 con 0x7f7bb0155ab0 2024-09-17T03:14:49.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.334+0000 7f7bb6c27640 1 -- 172.21.15.87:0/2865415882 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bb015c8f0 con 0x7f7bb0155ab0 2024-09-17T03:14:49.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.335+0000 7f7bad7fa640 1 -- 172.21.15.87:0/2865415882 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7ba400ac50 con 0x7f7bb0155ab0 2024-09-17T03:14:49.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.335+0000 7f7bad7fa640 1 --2- 172.21.15.87:0/2865415882 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7b9c078b90 0x7f7b9c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:49.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.335+0000 7f7bad7fa640 1 -- 172.21.15.87:0/2865415882 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4943+0+0 (secure 0 0 0) 0x7f7ba409a940 con 0x7f7bb0155ab0 2024-09-17T03:14:49.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.336+0000 7f7bb49c3640 1 --2- 172.21.15.87:0/2865415882 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7b9c078b90 0x7f7b9c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:49.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.336+0000 7f7bb49c3640 1 --2- 172.21.15.87:0/2865415882 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7b9c078b90 0x7f7b9c07b0a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0033000 tx=0x7f7ba0004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:49.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.340+0000 7f7bad7fa640 1 -- 172.21.15.87:0/2865415882 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7bb0147dc0 con 0x7f7bb0155ab0 2024-09-17T03:14:49.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:49.525+0000 7f7bb6c27640 1 -- 172.21.15.87:0/2865415882 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f7bb00ccff0 con 0x7f7bb0155ab0 2024-09-17T03:14:50.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.141+0000 7f7bad7fa640 1 -- 172.21.15.87:0/2865415882 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) ==== 97+0+0 (secure 0 0 0) 0x7f7ba4016020 con 0x7f7bb0155ab0 2024-09-17T03:14:50.141 INFO:teuthology.orchestra.run.smithi087.stderr: removed all blocklist entries 2024-09-17T03:14:50.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.144+0000 7f7bb6c27640 1 -- 172.21.15.87:0/2865415882 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7b9c078b90 msgr2=0x7f7b9c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:50.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.144+0000 7f7bb6c27640 1 --2- 172.21.15.87:0/2865415882 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7b9c078b90 0x7f7b9c07b0a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0033000 tx=0x7f7ba0004730 comp rx=0 tx=0).stop 2024-09-17T03:14:50.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.144+0000 7f7bb6c27640 1 -- 172.21.15.87:0/2865415882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bb0155ab0 msgr2=0x7f7bb00b0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:50.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.144+0000 7f7bb6c27640 1 --2- 172.21.15.87:0/2865415882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bb0155ab0 0x7f7bb00b0710 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400c7b0 tx=0x7f7ba400cc70 comp rx=0 tx=0).stop 2024-09-17T03:14:50.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.144+0000 7f7bb6c27640 1 -- 172.21.15.87:0/2865415882 shutdown_connections 2024-09-17T03:14:50.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.144+0000 7f7bb6c27640 1 --2- 172.21.15.87:0/2865415882 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7b9c078b90 0x7f7b9c07b0a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:50.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.144+0000 7f7bb6c27640 1 --2- 172.21.15.87:0/2865415882 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7bb01418a0 0x7f7bb00b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:50.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.144+0000 7f7bb6c27640 1 --2- 172.21.15.87:0/2865415882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bb0155ab0 0x7f7bb00b0710 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:50.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.144+0000 7f7bb6c27640 1 --2- 172.21.15.87:0/2865415882 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7bb0071ed0 0x7f7bb0147880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:50.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.144+0000 7f7bb6c27640 1 -- 172.21.15.87:0/2865415882 >> 172.21.15.87:0/2865415882 conn(0x7f7bb00b2d10 msgr2=0x7f7bb00d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:50.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.144+0000 7f7bb6c27640 1 -- 172.21.15.87:0/2865415882 shutdown_connections 2024-09-17T03:14:50.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.144+0000 7f7bb6c27640 1 -- 172.21.15.87:0/2865415882 wait complete. 2024-09-17T03:14:50.157 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T03:14:50.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.271+0000 7f0c4c884640 1 -- 172.21.15.87:0/3556572842 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c34004000 msgr2=0x7f0c34004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:50.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.271+0000 7f0c4c884640 1 --2- 172.21.15.87:0/3556572842 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c34004000 0x7f0c34004800 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0c300016c0 tx=0x7f0c34005830 comp rx=0 tx=0).stop 2024-09-17T03:14:50.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.271+0000 7f0c46d9d640 1 -- 172.21.15.87:0/3556572842 reap_dead start 2024-09-17T03:14:50.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.271+0000 7f0c4c884640 1 -- 172.21.15.87:0/3556572842 shutdown_connections 2024-09-17T03:14:50.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.271+0000 7f0c4c884640 1 -- 172.21.15.87:0/3556572842 >> 172.21.15.87:0/3556572842 conn(0x7f0c480d1110 msgr2=0x7f0c480d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:50.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.271+0000 7f0c4c884640 1 -- 172.21.15.87:0/3556572842 shutdown_connections 2024-09-17T03:14:50.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.271+0000 7f0c4c884640 1 -- 172.21.15.87:0/3556572842 wait complete. 2024-09-17T03:14:50.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.273+0000 7f0c4c884640 1 Processor -- start 2024-09-17T03:14:50.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.273+0000 7f0c4c884640 1 -- start start 2024-09-17T03:14:50.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.274+0000 7f0c4c884640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c34004000 0x7f0c480ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:50.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.274+0000 7f0c4c884640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c48155600 0x7f0c48158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:50.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.274+0000 7f0c4c884640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c48155b50 0x7f0c48159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:50.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.274+0000 7f0c4c884640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4815f400 con 0x7f0c48155b50 2024-09-17T03:14:50.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.274+0000 7f0c4c884640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4815f280 con 0x7f0c34004000 2024-09-17T03:14:50.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.274+0000 7f0c4c884640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0c480cc4a0 con 0x7f0c48155600 2024-09-17T03:14:50.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.274+0000 7f0c46d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c48155b50 0x7f0c48159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:50.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.274+0000 7f0c46d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c48155b50 0x7f0c48159350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34140/0 (socket says 172.21.15.87:34140) 2024-09-17T03:14:50.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.274+0000 7f0c46d9d640 1 -- 172.21.15.87:0/2410015974 learned_addr learned my addr 172.21.15.87:0/2410015974 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:50.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.274+0000 7f0c4659c640 1 --2- 172.21.15.87:0/2410015974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c34004000 0x7f0c480ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:50.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.274+0000 7f0c3dd9b640 1 --2- 172.21.15.87:0/2410015974 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c48155600 0x7f0c48158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:50.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.275+0000 7f0c46d9d640 1 -- 172.21.15.87:0/2410015974 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c48155600 msgr2=0x7f0c48158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:50.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.275+0000 7f0c46d9d640 1 --2- 172.21.15.87:0/2410015974 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c48155600 0x7f0c48158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:50.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.275+0000 7f0c46d9d640 1 -- 172.21.15.87:0/2410015974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c34004000 msgr2=0x7f0c480ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:50.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.275+0000 7f0c46d9d640 1 --2- 172.21.15.87:0/2410015974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c34004000 0x7f0c480ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:50.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.275+0000 7f0c46d9d640 1 -- 172.21.15.87:0/2410015974 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c480b3810 con 0x7f0c48155b50 2024-09-17T03:14:50.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.275+0000 7f0c46d9d640 1 --2- 172.21.15.87:0/2410015974 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c48155b50 0x7f0c48159350 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0c4809ebd0 tx=0x7f0c40006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:50.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.275+0000 7f0c3f7fe640 1 -- 172.21.15.87:0/2410015974 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4000fa80 con 0x7f0c48155b50 2024-09-17T03:14:50.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.275+0000 7f0c3f7fe640 1 -- 172.21.15.87:0/2410015974 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c40018070 con 0x7f0c48155b50 2024-09-17T03:14:50.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.275+0000 7f0c3f7fe640 1 -- 172.21.15.87:0/2410015974 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4000fea0 con 0x7f0c48155b50 2024-09-17T03:14:50.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.275+0000 7f0c4c884640 1 -- 172.21.15.87:0/2410015974 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c480b3af0 con 0x7f0c48155b50 2024-09-17T03:14:50.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.275+0000 7f0c4c884640 1 -- 172.21.15.87:0/2410015974 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c480b5890 con 0x7f0c48155b50 2024-09-17T03:14:50.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.276+0000 7f0c4c884640 1 -- 172.21.15.87:0/2410015974 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c4815c220 con 0x7f0c48155b50 2024-09-17T03:14:50.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.277+0000 7f0c3f7fe640 1 -- 172.21.15.87:0/2410015974 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0c4001d460 con 0x7f0c48155b50 2024-09-17T03:14:50.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.277+0000 7f0c3f7fe640 1 --2- 172.21.15.87:0/2410015974 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0c28078b90 0x7f0c2807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:50.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.277+0000 7f0c3f7fe640 1 -- 172.21.15.87:0/2410015974 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f0c4009b120 con 0x7f0c48155b50 2024-09-17T03:14:50.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.278+0000 7f0c4659c640 1 --2- 172.21.15.87:0/2410015974 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0c28078b90 0x7f0c2807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:50.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.278+0000 7f0c4659c640 1 --2- 172.21.15.87:0/2410015974 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0c28078b90 0x7f0c2807b0a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009c50 tx=0x7f0c38005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:50.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.282+0000 7f0c3f7fe640 1 -- 172.21.15.87:0/2410015974 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0c400638b0 con 0x7f0c48155b50 2024-09-17T03:14:50.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.519+0000 7f0c4c884640 1 -- 172.21.15.87:0/2410015974 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f0c4815c5a0 con 0x7f0c48155b50 2024-09-17T03:14:50.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.520+0000 7f0c3f7fe640 1 -- 172.21.15.87:0/2410015974 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+6645 (secure 0 0 0) 0x7f0c400631b0 con 0x7f0c48155b50 2024-09-17T03:14:50.520 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:50.521 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-17T03:14:50.521 INFO:teuthology.orchestra.run.smithi087.stdout: "auth_dump": [ 2024-09-17T03:14:50.521 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:50.521 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.a", 2024-09-17T03:14:50.521 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDH8+hmLoKqFBAAl7DDFJqmrUwbDOL3J8wysA==", 2024-09-17T03:14:50.521 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:14:50.521 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-09-17T03:14:50.521 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-09-17T03:14:50.521 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-09-17T03:14:50.521 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:14:50.521 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.522 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:50.522 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:50.522 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.b", 2024-09-17T03:14:50.522 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDH8+hmOziYIRAA2hVbfPg5KtY+dF/o1HzAFA==", 2024-09-17T03:14:50.522 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:14:50.522 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-09-17T03:14:50.522 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-09-17T03:14:50.522 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-09-17T03:14:50.522 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:14:50.522 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.522 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:50.522 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:50.522 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.c", 2024-09-17T03:14:50.522 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDH8+hm+QtNHBAANIlkESYiJ80tcoyvu5KZLA==", 2024-09-17T03:14:50.523 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:14:50.523 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-09-17T03:14:50.523 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-09-17T03:14:50.523 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-09-17T03:14:50.523 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:14:50.523 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.523 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:50.523 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:50.523 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.d", 2024-09-17T03:14:50.523 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDH8+hmRUxHKRAARWfxi+PE89hD4fMhCdbnxQ==", 2024-09-17T03:14:50.524 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:14:50.524 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-09-17T03:14:50.524 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-09-17T03:14:50.524 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-09-17T03:14:50.524 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:14:50.524 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.524 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:50.524 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:50.524 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.0", 2024-09-17T03:14:50.524 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDL8+hm7Rx0ChAA8g5yLI7cCaX55jkLjNkzQA==", 2024-09-17T03:14:50.524 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:14:50.524 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-09-17T03:14:50.525 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-09-17T03:14:50.525 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:14:50.525 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.525 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:50.525 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:50.525 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.1", 2024-09-17T03:14:50.525 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDO8+hmQzq8KxAA5LsvmZb7P2FIPXd9wc23Ig==", 2024-09-17T03:14:50.525 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:14:50.525 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-09-17T03:14:50.525 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-09-17T03:14:50.525 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:14:50.525 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:50.526 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:50.526 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.2", 2024-09-17T03:14:50.526 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDS8+hm0jabEBAAYJIY3OgCoJ4EcYBJr89xSQ==", 2024-09-17T03:14:50.526 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:14:50.526 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-09-17T03:14:50.526 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-09-17T03:14:50.526 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:14:50.526 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:50.526 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:50.526 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.3", 2024-09-17T03:14:50.526 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDV8+hmSrsMIhAAWW1HvxXttwHNUJT5NxsPYA==", 2024-09-17T03:14:50.526 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:14:50.526 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-09-17T03:14:50.527 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-09-17T03:14:50.527 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:14:50.527 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.527 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:50.527 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:50.527 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.4", 2024-09-17T03:14:50.527 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDb8+hmg54IOxAAKQ8plXDnZxp7QlGZcNOVXg==", 2024-09-17T03:14:50.527 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:14:50.527 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-09-17T03:14:50.527 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-09-17T03:14:50.527 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:14:50.527 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.527 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:50.528 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:50.528 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.5", 2024-09-17T03:14:50.528 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDf8+hmUT1GLhAA6NQyWAThxgYjnf2X+PFE0A==", 2024-09-17T03:14:50.528 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:14:50.528 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-09-17T03:14:50.528 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-09-17T03:14:50.528 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:14:50.528 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.528 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:50.528 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:50.528 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.6", 2024-09-17T03:14:50.529 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDj8+hmjDo2IRAA6UyXPCnTHBilelz+S6IWmQ==", 2024-09-17T03:14:50.529 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:14:50.529 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-09-17T03:14:50.529 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-09-17T03:14:50.529 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:14:50.529 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.529 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:50.529 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:50.529 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.7", 2024-09-17T03:14:50.529 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDm8+hmJKtHMhAA56cm6L9B9GLL48bRfjBnLg==", 2024-09-17T03:14:50.529 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:14:50.529 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-09-17T03:14:50.530 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-09-17T03:14:50.530 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:14:50.530 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.530 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:50.530 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:50.530 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.0", 2024-09-17T03:14:50.530 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDH8+hmLskNLhAAd2M5ynGP8W6Pim11F18wYA==", 2024-09-17T03:14:50.530 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:14:50.530 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-09-17T03:14:50.530 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-09-17T03:14:50.530 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-09-17T03:14:50.531 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-09-17T03:14:50.531 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.531 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:50.531 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:50.531 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.1", 2024-09-17T03:14:50.531 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDH8+hm+aGqNRAAUG+G7uzrYQ90DHV08QvWKQ==", 2024-09-17T03:14:50.531 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:14:50.531 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-09-17T03:14:50.531 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-09-17T03:14:50.531 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-09-17T03:14:50.531 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-09-17T03:14:50.532 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.532 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:50.532 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:50.532 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.2", 2024-09-17T03:14:50.532 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDH8+hmqYfNMRAAzS0etgXaryNA6diH8MM4vQ==", 2024-09-17T03:14:50.532 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:14:50.532 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-09-17T03:14:50.532 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-09-17T03:14:50.532 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-09-17T03:14:50.532 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-09-17T03:14:50.532 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.532 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:50.532 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:50.533 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.3", 2024-09-17T03:14:50.533 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDH8+hmKQl7ORAAV04v5s/X/fn24bp8kLIRsA==", 2024-09-17T03:14:50.533 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:14:50.533 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-09-17T03:14:50.533 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-09-17T03:14:50.533 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-09-17T03:14:50.533 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-09-17T03:14:50.533 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.533 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:50.533 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:50.533 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.admin", 2024-09-17T03:14:50.534 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDG8+hmlf1wMhAA0b5O35T4hJS7YPD/EBn0WQ==", 2024-09-17T03:14:50.534 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:14:50.534 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-09-17T03:14:50.534 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-09-17T03:14:50.534 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-09-17T03:14:50.534 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:14:50.534 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.534 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:50.534 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:50.534 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.bootstrap-mds", 2024-09-17T03:14:50.534 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDs8+hm4BnhOBAAuAsw7+8jCwWDGNW0HuJ0Fg==", 2024-09-17T03:14:50.534 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:14:50.534 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile bootstrap-mds" 2024-09-17T03:14:50.534 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.535 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:50.535 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:50.535 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.bootstrap-mgr", 2024-09-17T03:14:50.535 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDs8+hmwSHhOBAAJPvUBfnOtuPl3SuJrmb05g==", 2024-09-17T03:14:50.535 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:14:50.535 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-17T03:14:50.535 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.535 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:50.535 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:50.535 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.bootstrap-osd", 2024-09-17T03:14:50.535 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDs8+hmxijhOBAAZrUMeAHBb93iFYj4IsPJKA==", 2024-09-17T03:14:50.536 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:14:50.536 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile bootstrap-osd" 2024-09-17T03:14:50.536 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.536 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:50.536 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:50.536 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.bootstrap-rbd", 2024-09-17T03:14:50.536 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDs8+hmni/hOBAAaiVmTl1oDjbuXT/QINa1Og==", 2024-09-17T03:14:50.536 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:14:50.536 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-17T03:14:50.536 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.536 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:50.536 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:50.536 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-17T03:14:50.537 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDs8+hmnzbhOBAAM5pzq1oaJTKnfHZvF0kqtg==", 2024-09-17T03:14:50.537 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:14:50.537 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-17T03:14:50.537 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.537 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:50.537 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:50.537 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.bootstrap-rgw", 2024-09-17T03:14:50.537 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDs8+hm4D3hOBAAiKntcD7k3zkqSn8Ups3KLg==", 2024-09-17T03:14:50.537 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:14:50.537 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-17T03:14:50.537 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.537 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:50.537 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:50.537 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.x", 2024-09-17T03:14:50.538 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDH8+hmGA/bEBAA7l5te6R2YeXND6keGQpVSw==", 2024-09-17T03:14:50.538 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:14:50.538 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-09-17T03:14:50.538 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-09-17T03:14:50.538 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:14:50.538 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.538 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:14:50.538 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:50.538 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.y", 2024-09-17T03:14:50.538 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDH8+hmsBkdDBAA4bwvaWSWANblx5JF75wTQQ==", 2024-09-17T03:14:50.538 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:14:50.538 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-09-17T03:14:50.538 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-09-17T03:14:50.539 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:14:50.539 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.539 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:50.539 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-17T03:14:50.539 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-17T03:14:50.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.522+0000 7f0c4c884640 1 -- 172.21.15.87:0/2410015974 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0c28078b90 msgr2=0x7f0c2807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:50.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.522+0000 7f0c4c884640 1 --2- 172.21.15.87:0/2410015974 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0c28078b90 0x7f0c2807b0a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009c50 tx=0x7f0c38005f90 comp rx=0 tx=0).stop 2024-09-17T03:14:50.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.522+0000 7f0c4c884640 1 -- 172.21.15.87:0/2410015974 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c48155b50 msgr2=0x7f0c48159350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:50.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.522+0000 7f0c4c884640 1 --2- 172.21.15.87:0/2410015974 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c48155b50 0x7f0c48159350 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0c4809ebd0 tx=0x7f0c40006660 comp rx=0 tx=0).stop 2024-09-17T03:14:50.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.522+0000 7f0c4c884640 1 -- 172.21.15.87:0/2410015974 shutdown_connections 2024-09-17T03:14:50.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.522+0000 7f0c4c884640 1 --2- 172.21.15.87:0/2410015974 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0c28078b90 0x7f0c2807b0a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:50.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.522+0000 7f0c4c884640 1 --2- 172.21.15.87:0/2410015974 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c48155b50 0x7f0c48159350 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:50.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.522+0000 7f0c4c884640 1 --2- 172.21.15.87:0/2410015974 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0c48155600 0x7f0c48158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:50.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.522+0000 7f0c4c884640 1 --2- 172.21.15.87:0/2410015974 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0c34004000 0x7f0c480ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:50.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.522+0000 7f0c4c884640 1 -- 172.21.15.87:0/2410015974 >> 172.21.15.87:0/2410015974 conn(0x7f0c480d1110 msgr2=0x7f0c4815a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:50.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.522+0000 7f0c4c884640 1 -- 172.21.15.87:0/2410015974 shutdown_connections 2024-09-17T03:14:50.542 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.522+0000 7f0c4c884640 1 -- 172.21.15.87:0/2410015974 wait complete. 2024-09-17T03:14:50.542 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-17T03:14:50.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.658+0000 7fd0df484640 1 -- 172.21.15.87:0/2973522573 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0d815ecf0 msgr2=0x7fd0d815f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:50.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.658+0000 7fd0df484640 1 --2- 172.21.15.87:0/2973522573 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0d815ecf0 0x7fd0d815f100 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8009830 tx=0x7fd0c802a510 comp rx=0 tx=0).stop 2024-09-17T03:14:50.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.658+0000 7fd0df484640 1 -- 172.21.15.87:0/2973522573 shutdown_connections 2024-09-17T03:14:50.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.658+0000 7fd0df484640 1 --2- 172.21.15.87:0/2973522573 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0d815ecf0 0x7fd0d815f100 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:50.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.658+0000 7fd0df484640 1 --2- 172.21.15.87:0/2973522573 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd0d815c2d0 0x7fd0d80cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:50.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.658+0000 7fd0df484640 1 -- 172.21.15.87:0/2973522573 >> 172.21.15.87:0/2973522573 conn(0x7fd0d80d4470 msgr2=0x7fd0d80d4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:50.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.658+0000 7fd0df484640 1 -- 172.21.15.87:0/2973522573 shutdown_connections 2024-09-17T03:14:50.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.658+0000 7fd0df484640 1 -- 172.21.15.87:0/2973522573 wait complete. 2024-09-17T03:14:50.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.660+0000 7fd0df484640 1 Processor -- start 2024-09-17T03:14:50.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.660+0000 7fd0df484640 1 -- start start 2024-09-17T03:14:50.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.661+0000 7fd0df484640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0d80b5ef0 0x7fd0d81490c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:50.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.661+0000 7fd0df484640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0d815c2d0 0x7fd0d814bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:50.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.661+0000 7fd0df484640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd0d815ecf0 0x7fd0d814c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:50.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.661+0000 7fd0df484640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d80d2db0 con 0x7fd0d815c2d0 2024-09-17T03:14:50.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.661+0000 7fd0df484640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd0d80d2c30 con 0x7fd0d80b5ef0 2024-09-17T03:14:50.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.661+0000 7fd0df484640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd0d80d2f30 con 0x7fd0d815ecf0 2024-09-17T03:14:50.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.661+0000 7fd0dca1f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0d815c2d0 0x7fd0d814bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:50.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.661+0000 7fd0dca1f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0d815c2d0 0x7fd0d814bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34164/0 (socket says 172.21.15.87:34164) 2024-09-17T03:14:50.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.661+0000 7fd0dca1f640 1 -- 172.21.15.87:0/1200557703 learned_addr learned my addr 172.21.15.87:0/1200557703 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:50.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.661+0000 7fd0dd220640 1 --2- 172.21.15.87:0/1200557703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0d80b5ef0 0x7fd0d81490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:50.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.661+0000 7fd0dda21640 1 --2- 172.21.15.87:0/1200557703 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd0d815ecf0 0x7fd0d814c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:50.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.661+0000 7fd0dca1f640 1 -- 172.21.15.87:0/1200557703 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd0d815ecf0 msgr2=0x7fd0d814c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:50.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.661+0000 7fd0dca1f640 1 --2- 172.21.15.87:0/1200557703 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd0d815ecf0 0x7fd0d814c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:50.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.661+0000 7fd0dca1f640 1 -- 172.21.15.87:0/1200557703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0d80b5ef0 msgr2=0x7fd0d81490c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:50.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.661+0000 7fd0dca1f640 1 --2- 172.21.15.87:0/1200557703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0d80b5ef0 0x7fd0d81490c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:50.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.661+0000 7fd0dca1f640 1 -- 172.21.15.87:0/1200557703 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0d80b9660 con 0x7fd0d815c2d0 2024-09-17T03:14:50.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.661+0000 7fd0dda21640 1 --2- 172.21.15.87:0/1200557703 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd0d815ecf0 0x7fd0d814c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:50.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.661+0000 7fd0dca1f640 1 --2- 172.21.15.87:0/1200557703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0d815c2d0 0x7fd0d814bb50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8009950 tx=0x7fd0c8033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:50.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.662+0000 7fd0cdffb640 1 -- 172.21.15.87:0/1200557703 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8031d60 con 0x7fd0d815c2d0 2024-09-17T03:14:50.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.662+0000 7fd0cdffb640 1 -- 172.21.15.87:0/1200557703 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0c8033e10 con 0x7fd0d815c2d0 2024-09-17T03:14:50.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.662+0000 7fd0df484640 1 -- 172.21.15.87:0/1200557703 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0d80b9940 con 0x7fd0d815c2d0 2024-09-17T03:14:50.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.662+0000 7fd0cdffb640 1 -- 172.21.15.87:0/1200557703 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8005260 con 0x7fd0d815c2d0 2024-09-17T03:14:50.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.662+0000 7fd0df484640 1 -- 172.21.15.87:0/1200557703 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0d80caea0 con 0x7fd0d815c2d0 2024-09-17T03:14:50.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.663+0000 7fd0df484640 1 -- 172.21.15.87:0/1200557703 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0d80d34c0 con 0x7fd0d815c2d0 2024-09-17T03:14:50.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.664+0000 7fd0cdffb640 1 -- 172.21.15.87:0/1200557703 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd0c8005400 con 0x7fd0d815c2d0 2024-09-17T03:14:50.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.664+0000 7fd0cdffb640 1 --2- 172.21.15.87:0/1200557703 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd0bc078b90 0x7fd0bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:50.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.664+0000 7fd0cdffb640 1 -- 172.21.15.87:0/1200557703 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fd0c80bbb70 con 0x7fd0d815c2d0 2024-09-17T03:14:50.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.664+0000 7fd0dd220640 1 --2- 172.21.15.87:0/1200557703 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd0bc078b90 0x7fd0bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:50.666 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.665+0000 7fd0dd220640 1 --2- 172.21.15.87:0/1200557703 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd0bc078b90 0x7fd0bc07b0a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400bb50 tx=0x7fd0d400b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:50.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.669+0000 7fd0cdffb640 1 -- 172.21.15.87:0/1200557703 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd0c803f030 con 0x7fd0d815c2d0 2024-09-17T03:14:50.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.910+0000 7fd0df484640 1 -- 172.21.15.87:0/1200557703 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fd0d815dc20 con 0x7fd0d815c2d0 2024-09-17T03:14:50.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.913+0000 7fd0cdffb640 1 -- 172.21.15.87:0/1200557703 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7fd0c8005d00 con 0x7fd0d815c2d0 2024-09-17T03:14:50.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.915+0000 7fd0df484640 1 -- 172.21.15.87:0/1200557703 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd0bc078b90 msgr2=0x7fd0bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:50.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.915+0000 7fd0df484640 1 --2- 172.21.15.87:0/1200557703 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd0bc078b90 0x7fd0bc07b0a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd0d400bb50 tx=0x7fd0d400b2e0 comp rx=0 tx=0).stop 2024-09-17T03:14:50.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.915+0000 7fd0df484640 1 -- 172.21.15.87:0/1200557703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0d815c2d0 msgr2=0x7fd0d814bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:50.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.915+0000 7fd0df484640 1 --2- 172.21.15.87:0/1200557703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0d815c2d0 0x7fd0d814bb50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8009950 tx=0x7fd0c8033c20 comp rx=0 tx=0).stop 2024-09-17T03:14:50.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.915+0000 7fd0df484640 1 -- 172.21.15.87:0/1200557703 shutdown_connections 2024-09-17T03:14:50.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.915+0000 7fd0df484640 1 --2- 172.21.15.87:0/1200557703 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd0bc078b90 0x7fd0bc07b0a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:50.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.915+0000 7fd0df484640 1 --2- 172.21.15.87:0/1200557703 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd0d815ecf0 0x7fd0d814c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:50.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.915+0000 7fd0df484640 1 --2- 172.21.15.87:0/1200557703 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0d815c2d0 0x7fd0d814bb50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:50.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.915+0000 7fd0df484640 1 --2- 172.21.15.87:0/1200557703 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd0d80b5ef0 0x7fd0d81490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:50.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.915+0000 7fd0df484640 1 -- 172.21.15.87:0/1200557703 >> 172.21.15.87:0/1200557703 conn(0x7fd0d80d4470 msgr2=0x7fd0d815ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:50.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.915+0000 7fd0df484640 1 -- 172.21.15.87:0/1200557703 shutdown_connections 2024-09-17T03:14:50.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:50.915+0000 7fd0df484640 1 -- 172.21.15.87:0/1200557703 wait complete. 2024-09-17T03:14:50.926 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-17T03:14:51.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.039+0000 7f3438fc3640 1 -- 172.21.15.87:0/2565668287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3434070d20 msgr2=0x7f3434071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:51.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.039+0000 7f3438fc3640 1 --2- 172.21.15.87:0/2565668287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3434070d20 0x7f3434071130 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f342c005fd0 tx=0x7f342c02a5d0 comp rx=0 tx=0).stop 2024-09-17T03:14:51.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.039+0000 7f3438fc3640 1 -- 172.21.15.87:0/2565668287 shutdown_connections 2024-09-17T03:14:51.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.039+0000 7f3438fc3640 1 --2- 172.21.15.87:0/2565668287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3434070d20 0x7f3434071130 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:51.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.039+0000 7f3438fc3640 1 --2- 172.21.15.87:0/2565668287 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3434068d00 0x7f34340cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:51.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.039+0000 7f3438fc3640 1 -- 172.21.15.87:0/2565668287 >> 172.21.15.87:0/2565668287 conn(0x7f34340d08c0 msgr2=0x7f34340d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:51.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.039+0000 7f3438fc3640 1 -- 172.21.15.87:0/2565668287 shutdown_connections 2024-09-17T03:14:51.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.039+0000 7f3438fc3640 1 -- 172.21.15.87:0/2565668287 wait complete. 2024-09-17T03:14:51.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.041+0000 7f3438fc3640 1 Processor -- start 2024-09-17T03:14:51.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.041+0000 7f3438fc3640 1 -- start start 2024-09-17T03:14:51.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.042+0000 7f3438fc3640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3434068d00 0x7f343414f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:51.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.043+0000 7f3438fc3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3434070d20 0x7f343415c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:51.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.043+0000 7f3438fc3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34340cbea0 0x7f34340aec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:51.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.043+0000 7f3438fc3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3434071670 con 0x7f34340cbea0 2024-09-17T03:14:51.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.043+0000 7f3438fc3640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f34340717f0 con 0x7f3434070d20 2024-09-17T03:14:51.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.043+0000 7f3438fc3640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3434071970 con 0x7f3434068d00 2024-09-17T03:14:51.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.043+0000 7f3432d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34340cbea0 0x7f34340aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:51.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.043+0000 7f3432d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34340cbea0 0x7f34340aec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34188/0 (socket says 172.21.15.87:34188) 2024-09-17T03:14:51.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.043+0000 7f3432d9d640 1 -- 172.21.15.87:0/2336807439 learned_addr learned my addr 172.21.15.87:0/2336807439 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:51.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.043+0000 7f343259c640 1 --2- 172.21.15.87:0/2336807439 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3434068d00 0x7f343414f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:51.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.043+0000 7f3431d9b640 1 --2- 172.21.15.87:0/2336807439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3434070d20 0x7f343415c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:51.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.043+0000 7f3432d9d640 1 -- 172.21.15.87:0/2336807439 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3434068d00 msgr2=0x7f343414f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:51.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.043+0000 7f3432d9d640 1 --2- 172.21.15.87:0/2336807439 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3434068d00 0x7f343414f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:51.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.043+0000 7f3432d9d640 1 -- 172.21.15.87:0/2336807439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3434070d20 msgr2=0x7f343415c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:51.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.043+0000 7f3432d9d640 1 --2- 172.21.15.87:0/2336807439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3434070d20 0x7f343415c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:51.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.043+0000 7f3432d9d640 1 -- 172.21.15.87:0/2336807439 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34340adbe0 con 0x7f34340cbea0 2024-09-17T03:14:51.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.043+0000 7f3432d9d640 1 --2- 172.21.15.87:0/2336807439 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34340cbea0 0x7f34340aec80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f342c009230 tx=0x7f342c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:51.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.043+0000 7f3422ffd640 1 -- 172.21.15.87:0/2336807439 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c03e030 con 0x7f34340cbea0 2024-09-17T03:14:51.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.043+0000 7f3422ffd640 1 -- 172.21.15.87:0/2336807439 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f342c003c20 con 0x7f34340cbea0 2024-09-17T03:14:51.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.043+0000 7f3438fc3640 1 -- 172.21.15.87:0/2336807439 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34340adec0 con 0x7f34340cbea0 2024-09-17T03:14:51.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.043+0000 7f3422ffd640 1 -- 172.21.15.87:0/2336807439 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c035040 con 0x7f34340cbea0 2024-09-17T03:14:51.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.043+0000 7f3438fc3640 1 -- 172.21.15.87:0/2336807439 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34340ae480 con 0x7f34340cbea0 2024-09-17T03:14:51.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.044+0000 7f3438fc3640 1 -- 172.21.15.87:0/2336807439 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f343415a0b0 con 0x7f34340cbea0 2024-09-17T03:14:51.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.045+0000 7f3422ffd640 1 -- 172.21.15.87:0/2336807439 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f342c041030 con 0x7f34340cbea0 2024-09-17T03:14:51.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.046+0000 7f3422ffd640 1 --2- 172.21.15.87:0/2336807439 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3414078b90 0x7f341407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:51.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.046+0000 7f3422ffd640 1 -- 172.21.15.87:0/2336807439 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f342c0bb000 con 0x7f34340cbea0 2024-09-17T03:14:51.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.046+0000 7f343259c640 1 --2- 172.21.15.87:0/2336807439 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3414078b90 0x7f341407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:51.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.047+0000 7f343259c640 1 --2- 172.21.15.87:0/2336807439 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3414078b90 0x7f341407b0a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3424005fd0 tx=0x7f34240095e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:51.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.050+0000 7f3422ffd640 1 -- 172.21.15.87:0/2336807439 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f343415a0b0 con 0x7f34340cbea0 2024-09-17T03:14:51.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.294+0000 7f3438fc3640 1 -- 172.21.15.87:0/2336807439 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f34340cb960 con 0x7f34340cbea0 2024-09-17T03:14:51.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.297+0000 7f3422ffd640 1 -- 172.21.15.87:0/2336807439 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f342c0830a0 con 0x7f34340cbea0 2024-09-17T03:14:51.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.299+0000 7f3438fc3640 1 -- 172.21.15.87:0/2336807439 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3414078b90 msgr2=0x7f341407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:51.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.299+0000 7f3438fc3640 1 --2- 172.21.15.87:0/2336807439 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3414078b90 0x7f341407b0a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3424005fd0 tx=0x7f34240095e0 comp rx=0 tx=0).stop 2024-09-17T03:14:51.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.299+0000 7f3438fc3640 1 -- 172.21.15.87:0/2336807439 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34340cbea0 msgr2=0x7f34340aec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:51.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.299+0000 7f3438fc3640 1 --2- 172.21.15.87:0/2336807439 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34340cbea0 0x7f34340aec80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f342c009230 tx=0x7f342c003820 comp rx=0 tx=0).stop 2024-09-17T03:14:51.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.299+0000 7f3438fc3640 1 -- 172.21.15.87:0/2336807439 shutdown_connections 2024-09-17T03:14:51.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.299+0000 7f3438fc3640 1 --2- 172.21.15.87:0/2336807439 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3414078b90 0x7f341407b0a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:51.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.299+0000 7f3438fc3640 1 --2- 172.21.15.87:0/2336807439 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34340cbea0 0x7f34340aec80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:51.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.299+0000 7f3438fc3640 1 --2- 172.21.15.87:0/2336807439 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3434070d20 0x7f343415c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:51.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.299+0000 7f3438fc3640 1 --2- 172.21.15.87:0/2336807439 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3434068d00 0x7f343414f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:51.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.299+0000 7f3438fc3640 1 -- 172.21.15.87:0/2336807439 >> 172.21.15.87:0/2336807439 conn(0x7f34340d08c0 msgr2=0x7f3434159910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:51.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.299+0000 7f3438fc3640 1 -- 172.21.15.87:0/2336807439 shutdown_connections 2024-09-17T03:14:51.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.299+0000 7f3438fc3640 1 -- 172.21.15.87:0/2336807439 wait complete. 2024-09-17T03:14:51.310 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-17T03:14:51.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.423+0000 7f81b8f82640 1 -- 172.21.15.87:0/3961559434 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f819c004800 msgr2=0x7f819c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:51.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.423+0000 7f81b8f82640 1 --2- 172.21.15.87:0/3961559434 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f819c004800 0x7f819c004c10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f81a800c4a0 tx=0x7f819c005830 comp rx=0 tx=0).stop 2024-09-17T03:14:51.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.423+0000 7f81b2d9d640 1 -- 172.21.15.87:0/3961559434 reap_dead start 2024-09-17T03:14:51.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.423+0000 7f81b8f82640 1 -- 172.21.15.87:0/3961559434 shutdown_connections 2024-09-17T03:14:51.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.423+0000 7f81b8f82640 1 -- 172.21.15.87:0/3961559434 >> 172.21.15.87:0/3961559434 conn(0x7f81b40d1110 msgr2=0x7f81b40d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:51.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.423+0000 7f81b8f82640 1 -- 172.21.15.87:0/3961559434 shutdown_connections 2024-09-17T03:14:51.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.423+0000 7f81b8f82640 1 -- 172.21.15.87:0/3961559434 wait complete. 2024-09-17T03:14:51.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.425+0000 7f81b8f82640 1 Processor -- start 2024-09-17T03:14:51.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.425+0000 7f81b8f82640 1 -- start start 2024-09-17T03:14:51.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.426+0000 7f81b8f82640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f819c004800 0x7f81b40b0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:51.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.426+0000 7f81b8f82640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81b41567a0 0x7f81b40af770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:51.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.426+0000 7f81b8f82640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81b4156c80 0x7f81b40afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:51.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.426+0000 7f81b8f82640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f81b415a680 con 0x7f81b41567a0 2024-09-17T03:14:51.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.426+0000 7f81b8f82640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f81b415a500 con 0x7f81b4156c80 2024-09-17T03:14:51.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.426+0000 7f81b8f82640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f81b406dd10 con 0x7f819c004800 2024-09-17T03:14:51.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.426+0000 7f81b1d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81b41567a0 0x7f81b40af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:51.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.426+0000 7f81b1d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81b41567a0 0x7f81b40af770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34192/0 (socket says 172.21.15.87:34192) 2024-09-17T03:14:51.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.426+0000 7f81b1d9b640 1 -- 172.21.15.87:0/2481591594 learned_addr learned my addr 172.21.15.87:0/2481591594 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:51.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.426+0000 7f81b2d9d640 1 --2- 172.21.15.87:0/2481591594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81b4156c80 0x7f81b40afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:51.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.426+0000 7f81b259c640 1 --2- 172.21.15.87:0/2481591594 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f819c004800 0x7f81b40b0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:51.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.427+0000 7f81b1d9b640 1 -- 172.21.15.87:0/2481591594 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f819c004800 msgr2=0x7f81b40b0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:51.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.427+0000 7f81b1d9b640 1 --2- 172.21.15.87:0/2481591594 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f819c004800 0x7f81b40b0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:51.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.427+0000 7f81b1d9b640 1 -- 172.21.15.87:0/2481591594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81b4156c80 msgr2=0x7f81b40afcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:51.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.427+0000 7f81b1d9b640 1 --2- 172.21.15.87:0/2481591594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81b4156c80 0x7f81b40afcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:51.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.427+0000 7f81b1d9b640 1 -- 172.21.15.87:0/2481591594 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b40b6af0 con 0x7f81b41567a0 2024-09-17T03:14:51.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.427+0000 7f81b1d9b640 1 --2- 172.21.15.87:0/2481591594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81b41567a0 0x7f81b40af770 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f81a802b300 tx=0x7f81a8006e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:51.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.427+0000 7f81a2ffd640 1 -- 172.21.15.87:0/2481591594 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8006570 con 0x7f81b41567a0 2024-09-17T03:14:51.428 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.427+0000 7f81a2ffd640 1 -- 172.21.15.87:0/2481591594 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81a802b650 con 0x7f81b41567a0 2024-09-17T03:14:51.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.427+0000 7f81b8f82640 1 -- 172.21.15.87:0/2481591594 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b406fe70 con 0x7f81b41567a0 2024-09-17T03:14:51.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.427+0000 7f81a2ffd640 1 -- 172.21.15.87:0/2481591594 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a802baa0 con 0x7f81b41567a0 2024-09-17T03:14:51.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.427+0000 7f81b8f82640 1 -- 172.21.15.87:0/2481591594 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b40cb4b0 con 0x7f81b41567a0 2024-09-17T03:14:51.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.428+0000 7f81b8f82640 1 -- 172.21.15.87:0/2481591594 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b40cf170 con 0x7f81b41567a0 2024-09-17T03:14:51.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.429+0000 7f81a2ffd640 1 -- 172.21.15.87:0/2481591594 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f81a80035f0 con 0x7f81b41567a0 2024-09-17T03:14:51.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.429+0000 7f81a2ffd640 1 --2- 172.21.15.87:0/2481591594 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8194078bc0 0x7f819407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:51.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.430+0000 7f81a2ffd640 1 -- 172.21.15.87:0/2481591594 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f81a80ba9b0 con 0x7f81b41567a0 2024-09-17T03:14:51.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.430+0000 7f81b259c640 1 --2- 172.21.15.87:0/2481591594 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8194078bc0 0x7f819407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:51.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.430+0000 7f81b259c640 1 --2- 172.21.15.87:0/2481591594 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8194078bc0 0x7f819407b0d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f81a4005fd0 tx=0x7f81a4009500 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:51.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.434+0000 7f81a2ffd640 1 -- 172.21.15.87:0/2481591594 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f81b40cc0b0 con 0x7f81b41567a0 2024-09-17T03:14:51.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.676+0000 7f81b8f82640 1 -- 172.21.15.87:0/2481591594 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f81b40cf170 con 0x7f81b41567a0 2024-09-17T03:14:51.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.679+0000 7f81a2ffd640 1 -- 172.21.15.87:0/2481591594 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7f81a8082bd0 con 0x7f81b41567a0 2024-09-17T03:14:51.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.680+0000 7f81b8f82640 1 -- 172.21.15.87:0/2481591594 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8194078bc0 msgr2=0x7f819407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:51.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.680+0000 7f81b8f82640 1 --2- 172.21.15.87:0/2481591594 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8194078bc0 0x7f819407b0d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f81a4005fd0 tx=0x7f81a4009500 comp rx=0 tx=0).stop 2024-09-17T03:14:51.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.680+0000 7f81b8f82640 1 -- 172.21.15.87:0/2481591594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81b41567a0 msgr2=0x7f81b40af770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:51.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.681+0000 7f81b8f82640 1 --2- 172.21.15.87:0/2481591594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81b41567a0 0x7f81b40af770 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f81a802b300 tx=0x7f81a8006e60 comp rx=0 tx=0).stop 2024-09-17T03:14:51.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.681+0000 7f81b8f82640 1 -- 172.21.15.87:0/2481591594 shutdown_connections 2024-09-17T03:14:51.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.681+0000 7f81b8f82640 1 --2- 172.21.15.87:0/2481591594 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8194078bc0 0x7f819407b0d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:51.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.681+0000 7f81b8f82640 1 --2- 172.21.15.87:0/2481591594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f81b4156c80 0x7f81b40afcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:51.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.681+0000 7f81b8f82640 1 --2- 172.21.15.87:0/2481591594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81b41567a0 0x7f81b40af770 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:51.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.681+0000 7f81b8f82640 1 --2- 172.21.15.87:0/2481591594 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f819c004800 0x7f81b40b0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:51.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.681+0000 7f81b8f82640 1 -- 172.21.15.87:0/2481591594 >> 172.21.15.87:0/2481591594 conn(0x7f81b40d1110 msgr2=0x7f81b4159670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:51.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.681+0000 7f81b8f82640 1 -- 172.21.15.87:0/2481591594 shutdown_connections 2024-09-17T03:14:51.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.681+0000 7f81b8f82640 1 -- 172.21.15.87:0/2481591594 wait complete. 2024-09-17T03:14:51.691 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-17T03:14:51.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.804+0000 7f766e1ba640 1 -- 172.21.15.87:0/283444970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76680cb760 msgr2=0x7f76680cbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:51.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.804+0000 7f766e1ba640 1 --2- 172.21.15.87:0/283444970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76680cb760 0x7f76680cbb70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f76680d10e0 tx=0x7f766002a560 comp rx=0 tx=0).stop 2024-09-17T03:14:51.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.804+0000 7f766e1ba640 1 -- 172.21.15.87:0/283444970 shutdown_connections 2024-09-17T03:14:51.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.804+0000 7f766e1ba640 1 --2- 172.21.15.87:0/283444970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76680cb760 0x7f76680cbb70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:51.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.804+0000 7f766e1ba640 1 --2- 172.21.15.87:0/283444970 >> v2:172.21.15.149:3300/0 conn(0x7f76680b03d0 0x7f766815a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:51.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.804+0000 7f766e1ba640 1 -- 172.21.15.87:0/283444970 >> 172.21.15.87:0/283444970 conn(0x7f76680d1110 msgr2=0x7f76680d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:51.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.804+0000 7f766e1ba640 1 -- 172.21.15.87:0/283444970 shutdown_connections 2024-09-17T03:14:51.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.804+0000 7f766e1ba640 1 -- 172.21.15.87:0/283444970 wait complete. 2024-09-17T03:14:51.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.806+0000 7f766e1ba640 1 Processor -- start 2024-09-17T03:14:51.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.806+0000 7f766e1ba640 1 -- start start 2024-09-17T03:14:51.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.807+0000 7f766e1ba640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76680b03d0 0x7f7668158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:51.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.807+0000 7f766e1ba640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76680cb760 0x7f766809ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:51.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.807+0000 7f766e1ba640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76681602e0 0x7f76680aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:51.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.807+0000 7f766e1ba640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f766815d3c0 con 0x7f76680cb760 2024-09-17T03:14:51.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.807+0000 7f766e1ba640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f766815d240 con 0x7f76680b03d0 2024-09-17T03:14:51.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.807+0000 7f766e1ba640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f766815d540 con 0x7f76681602e0 2024-09-17T03:14:51.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.808+0000 7f7666ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76680cb760 0x7f766809ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:51.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.808+0000 7f7666ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76680cb760 0x7f766809ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34198/0 (socket says 172.21.15.87:34198) 2024-09-17T03:14:51.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.808+0000 7f7666ffd640 1 -- 172.21.15.87:0/2369501078 learned_addr learned my addr 172.21.15.87:0/2369501078 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:51.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.808+0000 7f76677fe640 1 --2- 172.21.15.87:0/2369501078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76680b03d0 0x7f7668158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:51.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.808+0000 7f7667fff640 1 --2- 172.21.15.87:0/2369501078 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76681602e0 0x7f76680aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:51.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.808+0000 7f7666ffd640 1 -- 172.21.15.87:0/2369501078 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76681602e0 msgr2=0x7f76680aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:51.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.808+0000 7f7666ffd640 1 --2- 172.21.15.87:0/2369501078 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76681602e0 0x7f76680aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:51.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.808+0000 7f7666ffd640 1 -- 172.21.15.87:0/2369501078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76680b03d0 msgr2=0x7f7668158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:51.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.808+0000 7f7666ffd640 1 --2- 172.21.15.87:0/2369501078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76680b03d0 0x7f7668158140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:51.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.808+0000 7f7666ffd640 1 -- 172.21.15.87:0/2369501078 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7668158680 con 0x7f76680cb760 2024-09-17T03:14:51.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.808+0000 7f7666ffd640 1 --2- 172.21.15.87:0/2369501078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76680cb760 0x7f766809ae40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f765000c9a0 tx=0x7f765000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:51.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.808+0000 7f765ffff640 1 -- 172.21.15.87:0/2369501078 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7650007bb0 con 0x7f76680cb760 2024-09-17T03:14:51.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.808+0000 7f766e1ba640 1 -- 172.21.15.87:0/2369501078 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f766809b900 con 0x7f76680cb760 2024-09-17T03:14:51.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.808+0000 7f765ffff640 1 -- 172.21.15.87:0/2369501078 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7650007e40 con 0x7f76680cb760 2024-09-17T03:14:51.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.808+0000 7f765ffff640 1 -- 172.21.15.87:0/2369501078 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765000f690 con 0x7f76680cb760 2024-09-17T03:14:51.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.808+0000 7f766e1ba640 1 -- 172.21.15.87:0/2369501078 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f766809d5b0 con 0x7f76680cb760 2024-09-17T03:14:51.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.809+0000 7f766e1ba640 1 -- 172.21.15.87:0/2369501078 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f766809b670 con 0x7f76680cb760 2024-09-17T03:14:51.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.810+0000 7f765ffff640 1 -- 172.21.15.87:0/2369501078 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f765000f830 con 0x7f76680cb760 2024-09-17T03:14:51.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.810+0000 7f765ffff640 1 --2- 172.21.15.87:0/2369501078 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7648078b90 0x7f764807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:51.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.811+0000 7f765ffff640 1 -- 172.21.15.87:0/2369501078 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f765009b120 con 0x7f76680cb760 2024-09-17T03:14:51.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.811+0000 7f76677fe640 1 --2- 172.21.15.87:0/2369501078 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7648078b90 0x7f764807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:51.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.811+0000 7f76677fe640 1 --2- 172.21.15.87:0/2369501078 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7648078b90 0x7f764807b0a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f765800b450 tx=0x7f765800d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:51.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:51.815+0000 7f765ffff640 1 -- 172.21.15.87:0/2369501078 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7668158680 con 0x7f76680cb760 2024-09-17T03:14:52.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.056+0000 7f766e1ba640 1 -- 172.21.15.87:0/2369501078 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f766815da60 con 0x7f76680cb760 2024-09-17T03:14:52.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.061+0000 7f765ffff640 1 -- 172.21.15.87:0/2369501078 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f7650014030 con 0x7f76680cb760 2024-09-17T03:14:52.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.063+0000 7f766e1ba640 1 -- 172.21.15.87:0/2369501078 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7648078b90 msgr2=0x7f764807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:52.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.063+0000 7f766e1ba640 1 --2- 172.21.15.87:0/2369501078 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7648078b90 0x7f764807b0a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f765800b450 tx=0x7f765800d040 comp rx=0 tx=0).stop 2024-09-17T03:14:52.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.063+0000 7f766e1ba640 1 -- 172.21.15.87:0/2369501078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76680cb760 msgr2=0x7f766809ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:52.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.063+0000 7f766e1ba640 1 --2- 172.21.15.87:0/2369501078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76680cb760 0x7f766809ae40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f765000c9a0 tx=0x7f765000ce60 comp rx=0 tx=0).stop 2024-09-17T03:14:52.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.063+0000 7f766e1ba640 1 -- 172.21.15.87:0/2369501078 shutdown_connections 2024-09-17T03:14:52.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.063+0000 7f766e1ba640 1 --2- 172.21.15.87:0/2369501078 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7648078b90 0x7f764807b0a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:52.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.063+0000 7f766e1ba640 1 --2- 172.21.15.87:0/2369501078 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f76681602e0 0x7f76680aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:52.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.063+0000 7f766e1ba640 1 --2- 172.21.15.87:0/2369501078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76680cb760 0x7f766809ae40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:52.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.063+0000 7f766e1ba640 1 --2- 172.21.15.87:0/2369501078 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f76680b03d0 0x7f7668158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:52.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.063+0000 7f766e1ba640 1 -- 172.21.15.87:0/2369501078 >> 172.21.15.87:0/2369501078 conn(0x7f76680d1110 msgr2=0x7f76680d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:52.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.063+0000 7f766e1ba640 1 -- 172.21.15.87:0/2369501078 shutdown_connections 2024-09-17T03:14:52.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.063+0000 7f766e1ba640 1 -- 172.21.15.87:0/2369501078 wait complete. 2024-09-17T03:14:52.074 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-17T03:14:52.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.188+0000 7efebd319640 1 -- 172.21.15.87:0/2150123577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efeb80d4150 msgr2=0x7efeb80d4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:52.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.188+0000 7efebd319640 1 --2- 172.21.15.87:0/2150123577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efeb80d4150 0x7efeb80d4560 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7efeb808c800 tx=0x7efeac02a4a0 comp rx=0 tx=0).stop 2024-09-17T03:14:52.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.188+0000 7efebd319640 1 -- 172.21.15.87:0/2150123577 shutdown_connections 2024-09-17T03:14:52.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.188+0000 7efebd319640 1 --2- 172.21.15.87:0/2150123577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efeb80d4150 0x7efeb80d4560 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:52.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.188+0000 7efebd319640 1 --2- 172.21.15.87:0/2150123577 >> v2:172.21.15.149:3300/0 conn(0x7efeb80d2c50 0x7efeb80d3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:52.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.188+0000 7efebd319640 1 -- 172.21.15.87:0/2150123577 >> 172.21.15.87:0/2150123577 conn(0x7efeb808dae0 msgr2=0x7efeb809d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:52.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.188+0000 7efebd319640 1 -- 172.21.15.87:0/2150123577 shutdown_connections 2024-09-17T03:14:52.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.188+0000 7efebd319640 1 -- 172.21.15.87:0/2150123577 wait complete. 2024-09-17T03:14:52.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.190+0000 7efebd319640 1 Processor -- start 2024-09-17T03:14:52.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.190+0000 7efebd319640 1 -- start start 2024-09-17T03:14:52.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.191+0000 7efebd319640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efeb80d0e60 0x7efeb80ca470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:52.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.191+0000 7efebd319640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efeb80d2c50 0x7efeb80ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:52.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.191+0000 7efebd319640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efeb80d4150 0x7efeb81f8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:52.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.191+0000 7efebd319640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efeb80d1980 con 0x7efeb80d2c50 2024-09-17T03:14:52.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.191+0000 7efebd319640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efeb81622b0 con 0x7efeb80d4150 2024-09-17T03:14:52.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.191+0000 7efebd319640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7efeb80d1b00 con 0x7efeb80d0e60 2024-09-17T03:14:52.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.191+0000 7efeb67fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efeb80d2c50 0x7efeb80ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:52.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.191+0000 7efeb67fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efeb80d2c50 0x7efeb80ca9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34216/0 (socket says 172.21.15.87:34216) 2024-09-17T03:14:52.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.191+0000 7efeb67fc640 1 -- 172.21.15.87:0/4013614907 learned_addr learned my addr 172.21.15.87:0/4013614907 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:52.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.191+0000 7efeb6ffd640 1 --2- 172.21.15.87:0/4013614907 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efeb80d0e60 0x7efeb80ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:52.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.191+0000 7efeb77fe640 1 --2- 172.21.15.87:0/4013614907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efeb80d4150 0x7efeb81f8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:52.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.191+0000 7efeb67fc640 1 -- 172.21.15.87:0/4013614907 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efeb80d0e60 msgr2=0x7efeb80ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:52.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.191+0000 7efeb67fc640 1 --2- 172.21.15.87:0/4013614907 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efeb80d0e60 0x7efeb80ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:52.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.191+0000 7efeb67fc640 1 -- 172.21.15.87:0/4013614907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efeb80d4150 msgr2=0x7efeb81f8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:52.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.191+0000 7efeb67fc640 1 --2- 172.21.15.87:0/4013614907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efeb80d4150 0x7efeb81f8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:52.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.191+0000 7efeb67fc640 1 -- 172.21.15.87:0/4013614907 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb814d3a0 con 0x7efeb80d2c50 2024-09-17T03:14:52.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.192+0000 7efeb67fc640 1 --2- 172.21.15.87:0/4013614907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efeb80d2c50 0x7efeb80ca9b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efeac009920 tx=0x7efeac033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:52.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.192+0000 7efe9f7fe640 1 -- 172.21.15.87:0/4013614907 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac031d60 con 0x7efeb80d2c50 2024-09-17T03:14:52.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.192+0000 7efe9f7fe640 1 -- 172.21.15.87:0/4013614907 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efeac005020 con 0x7efeb80d2c50 2024-09-17T03:14:52.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.192+0000 7efe9f7fe640 1 -- 172.21.15.87:0/4013614907 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac005470 con 0x7efeb80d2c50 2024-09-17T03:14:52.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.192+0000 7efebd319640 1 -- 172.21.15.87:0/4013614907 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb814d670 con 0x7efeb80d2c50 2024-09-17T03:14:52.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.192+0000 7efebd319640 1 -- 172.21.15.87:0/4013614907 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb815e3a0 con 0x7efeb80d2c50 2024-09-17T03:14:52.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.193+0000 7efebd319640 1 -- 172.21.15.87:0/4013614907 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb8150130 con 0x7efeb80d2c50 2024-09-17T03:14:52.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.194+0000 7efe9f7fe640 1 -- 172.21.15.87:0/4013614907 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efeac0047a0 con 0x7efeb80d2c50 2024-09-17T03:14:52.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.194+0000 7efe9f7fe640 1 --2- 172.21.15.87:0/4013614907 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7efea4078b90 0x7efea407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:52.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.194+0000 7efe9f7fe640 1 -- 172.21.15.87:0/4013614907 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7efeac0bbd60 con 0x7efeb80d2c50 2024-09-17T03:14:52.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.194+0000 7efeb6ffd640 1 --2- 172.21.15.87:0/4013614907 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7efea4078b90 0x7efea407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:52.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.195+0000 7efeb6ffd640 1 --2- 172.21.15.87:0/4013614907 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7efea4078b90 0x7efea407b0a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efea800b810 tx=0x7efea8008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:52.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.199+0000 7efe9f7fe640 1 -- 172.21.15.87:0/4013614907 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7efeac03f030 con 0x7efeb80d2c50 2024-09-17T03:14:52.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.442+0000 7efebd319640 1 -- 172.21.15.87:0/4013614907 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7efeb80d0150 con 0x7efeb80d2c50 2024-09-17T03:14:52.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.445+0000 7efe9f7fe640 1 -- 172.21.15.87:0/4013614907 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7efeac0844f0 con 0x7efeb80d2c50 2024-09-17T03:14:52.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.447+0000 7efebd319640 1 -- 172.21.15.87:0/4013614907 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7efea4078b90 msgr2=0x7efea407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:52.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.447+0000 7efebd319640 1 --2- 172.21.15.87:0/4013614907 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7efea4078b90 0x7efea407b0a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efea800b810 tx=0x7efea8008040 comp rx=0 tx=0).stop 2024-09-17T03:14:52.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.447+0000 7efebd319640 1 -- 172.21.15.87:0/4013614907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efeb80d2c50 msgr2=0x7efeb80ca9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:52.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.447+0000 7efebd319640 1 --2- 172.21.15.87:0/4013614907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efeb80d2c50 0x7efeb80ca9b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7efeac009920 tx=0x7efeac033d60 comp rx=0 tx=0).stop 2024-09-17T03:14:52.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.447+0000 7efebd319640 1 -- 172.21.15.87:0/4013614907 shutdown_connections 2024-09-17T03:14:52.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.447+0000 7efebd319640 1 --2- 172.21.15.87:0/4013614907 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7efea4078b90 0x7efea407b0a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:52.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.447+0000 7efebd319640 1 --2- 172.21.15.87:0/4013614907 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efeb80d4150 0x7efeb81f8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:52.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.447+0000 7efebd319640 1 --2- 172.21.15.87:0/4013614907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efeb80d2c50 0x7efeb80ca9b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:52.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.447+0000 7efebd319640 1 --2- 172.21.15.87:0/4013614907 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efeb80d0e60 0x7efeb80ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:52.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.447+0000 7efebd319640 1 -- 172.21.15.87:0/4013614907 >> 172.21.15.87:0/4013614907 conn(0x7efeb808dae0 msgr2=0x7efeb80b8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:52.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.447+0000 7efebd319640 1 -- 172.21.15.87:0/4013614907 shutdown_connections 2024-09-17T03:14:52.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.447+0000 7efebd319640 1 -- 172.21.15.87:0/4013614907 wait complete. 2024-09-17T03:14:52.458 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-17T03:14:52.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.571+0000 7f2cc758b640 1 -- 172.21.15.87:0/1070885356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cac004d90 msgr2=0x7f2cac005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:52.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.571+0000 7f2cc758b640 1 --2- 172.21.15.87:0/1070885356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cac004d90 0x7f2cac005220 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2cb800c610 tx=0x7f2cac005830 comp rx=0 tx=0).stop 2024-09-17T03:14:52.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.571+0000 7f2cc5b28640 1 -- 172.21.15.87:0/1070885356 reap_dead start 2024-09-17T03:14:52.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.571+0000 7f2cc758b640 1 -- 172.21.15.87:0/1070885356 shutdown_connections 2024-09-17T03:14:52.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.571+0000 7f2cc758b640 1 -- 172.21.15.87:0/1070885356 >> 172.21.15.87:0/1070885356 conn(0x7f2cc0069360 msgr2=0x7f2cc00ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:52.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.571+0000 7f2cc758b640 1 -- 172.21.15.87:0/1070885356 shutdown_connections 2024-09-17T03:14:52.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.571+0000 7f2cc758b640 1 -- 172.21.15.87:0/1070885356 wait complete. 2024-09-17T03:14:52.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.573+0000 7f2cc758b640 1 Processor -- start 2024-09-17T03:14:52.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.573+0000 7f2cc758b640 1 -- start start 2024-09-17T03:14:52.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.574+0000 7f2cc758b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cac004d90 0x7f2cc0151710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:52.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.574+0000 7f2cc758b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cc0153f20 0x7f2cc00710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:52.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.574+0000 7f2cc758b640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2cc0154330 0x7f2cc0071620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:52.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.574+0000 7f2cc758b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc006def0 con 0x7f2cc0153f20 2024-09-17T03:14:52.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.574+0000 7f2cc758b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc015c6e0 con 0x7f2cac004d90 2024-09-17T03:14:52.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.574+0000 7f2cc758b640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f2cc006e070 con 0x7f2cc0154330 2024-09-17T03:14:52.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.574+0000 7f2cc4b26640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cc0153f20 0x7f2cc00710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:52.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.574+0000 7f2cc4b26640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cc0153f20 0x7f2cc00710e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34234/0 (socket says 172.21.15.87:34234) 2024-09-17T03:14:52.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.574+0000 7f2cc4b26640 1 -- 172.21.15.87:0/3512709617 learned_addr learned my addr 172.21.15.87:0/3512709617 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:52.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.575+0000 7f2cc5b28640 1 --2- 172.21.15.87:0/3512709617 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2cc0154330 0x7f2cc0071620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:52.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.575+0000 7f2cc5327640 1 --2- 172.21.15.87:0/3512709617 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cac004d90 0x7f2cc0151710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:52.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.575+0000 7f2cc4b26640 1 -- 172.21.15.87:0/3512709617 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2cc0154330 msgr2=0x7f2cc0071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:52.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.575+0000 7f2cc4b26640 1 --2- 172.21.15.87:0/3512709617 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2cc0154330 0x7f2cc0071620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:52.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.575+0000 7f2cc4b26640 1 -- 172.21.15.87:0/3512709617 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cac004d90 msgr2=0x7f2cc0151710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:52.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.575+0000 7f2cc4b26640 1 --2- 172.21.15.87:0/3512709617 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cac004d90 0x7f2cc0151710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:52.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.575+0000 7f2cc4b26640 1 -- 172.21.15.87:0/3512709617 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc00b7eb0 con 0x7f2cc0153f20 2024-09-17T03:14:52.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.575+0000 7f2cc5b28640 1 --2- 172.21.15.87:0/3512709617 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2cc0154330 0x7f2cc0071620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:14:52.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.575+0000 7f2cc5327640 1 --2- 172.21.15.87:0/3512709617 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cac004d90 0x7f2cc0151710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:14:52.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.575+0000 7f2cc4b26640 1 --2- 172.21.15.87:0/3512709617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cc0153f20 0x7f2cc00710e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0004dc0 tx=0x7f2cb0001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:52.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.575+0000 7f2cb5ffb640 1 -- 172.21.15.87:0/3512709617 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0010890 con 0x7f2cc0153f20 2024-09-17T03:14:52.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.575+0000 7f2cb5ffb640 1 -- 172.21.15.87:0/3512709617 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cb0010b40 con 0x7f2cc0153f20 2024-09-17T03:14:52.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.575+0000 7f2cc758b640 1 -- 172.21.15.87:0/3512709617 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc01f5220 con 0x7f2cc0153f20 2024-09-17T03:14:52.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.575+0000 7f2cb5ffb640 1 -- 172.21.15.87:0/3512709617 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0011460 con 0x7f2cc0153f20 2024-09-17T03:14:52.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.575+0000 7f2cc758b640 1 -- 172.21.15.87:0/3512709617 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc01f5500 con 0x7f2cc0153f20 2024-09-17T03:14:52.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.576+0000 7f2cc758b640 1 -- 172.21.15.87:0/3512709617 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2cc006e580 con 0x7f2cc0153f20 2024-09-17T03:14:52.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.577+0000 7f2cb5ffb640 1 -- 172.21.15.87:0/3512709617 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2cb0011600 con 0x7f2cc0153f20 2024-09-17T03:14:52.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.577+0000 7f2cb5ffb640 1 --2- 172.21.15.87:0/3512709617 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f2ca4078bc0 0x7f2ca407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:52.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.577+0000 7f2cb5ffb640 1 -- 172.21.15.87:0/3512709617 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f2cb009b4b0 con 0x7f2cc0153f20 2024-09-17T03:14:52.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.577+0000 7f2cc5327640 1 --2- 172.21.15.87:0/3512709617 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f2ca4078bc0 0x7f2ca407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:52.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.578+0000 7f2cc5327640 1 --2- 172.21.15.87:0/3512709617 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f2ca4078bc0 0x7f2ca407b0d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc009b40 tx=0x7f2cbc005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:52.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.582+0000 7f2cb5ffb640 1 -- 172.21.15.87:0/3512709617 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f2cb0063cd0 con 0x7f2cc0153f20 2024-09-17T03:14:52.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.825+0000 7f2cc758b640 1 -- 172.21.15.87:0/3512709617 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f2cc00b2640 con 0x7f2cc0153f20 2024-09-17T03:14:52.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.827+0000 7f2cb5ffb640 1 -- 172.21.15.87:0/3512709617 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f2cb00635d0 con 0x7f2cc0153f20 2024-09-17T03:14:52.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.830+0000 7f2cc758b640 1 -- 172.21.15.87:0/3512709617 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f2ca4078bc0 msgr2=0x7f2ca407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:52.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.830+0000 7f2cc758b640 1 --2- 172.21.15.87:0/3512709617 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f2ca4078bc0 0x7f2ca407b0d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc009b40 tx=0x7f2cbc005f90 comp rx=0 tx=0).stop 2024-09-17T03:14:52.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.830+0000 7f2cc758b640 1 -- 172.21.15.87:0/3512709617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cc0153f20 msgr2=0x7f2cc00710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:52.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.830+0000 7f2cc758b640 1 --2- 172.21.15.87:0/3512709617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cc0153f20 0x7f2cc00710e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0004dc0 tx=0x7f2cb0001800 comp rx=0 tx=0).stop 2024-09-17T03:14:52.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.831+0000 7f2cc758b640 1 -- 172.21.15.87:0/3512709617 shutdown_connections 2024-09-17T03:14:52.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.831+0000 7f2cc758b640 1 --2- 172.21.15.87:0/3512709617 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f2ca4078bc0 0x7f2ca407b0d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:52.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.831+0000 7f2cc758b640 1 --2- 172.21.15.87:0/3512709617 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f2cc0154330 0x7f2cc0071620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:52.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.831+0000 7f2cc758b640 1 --2- 172.21.15.87:0/3512709617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2cc0153f20 0x7f2cc00710e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:52.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.831+0000 7f2cc758b640 1 --2- 172.21.15.87:0/3512709617 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f2cac004d90 0x7f2cc0151710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:52.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.831+0000 7f2cc758b640 1 -- 172.21.15.87:0/3512709617 >> 172.21.15.87:0/3512709617 conn(0x7f2cc0069360 msgr2=0x7f2cc00b1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:52.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.831+0000 7f2cc758b640 1 -- 172.21.15.87:0/3512709617 shutdown_connections 2024-09-17T03:14:52.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.831+0000 7f2cc758b640 1 -- 172.21.15.87:0/3512709617 wait complete. 2024-09-17T03:14:52.842 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T03:14:52.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.953+0000 7feec714a640 1 -- 172.21.15.87:0/2423091044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feec00cd690 msgr2=0x7feec00d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:52.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.953+0000 7feec714a640 1 --2- 172.21.15.87:0/2423091044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feec00cd690 0x7feec00d0f50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7feeb8005fd0 tx=0x7feeb802a8b0 comp rx=0 tx=0).stop 2024-09-17T03:14:52.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.953+0000 7feec714a640 1 -- 172.21.15.87:0/2423091044 shutdown_connections 2024-09-17T03:14:52.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.953+0000 7feec714a640 1 --2- 172.21.15.87:0/2423091044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feec00cd690 0x7feec00d0f50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:52.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.953+0000 7feec714a640 1 -- 172.21.15.87:0/2423091044 >> 172.21.15.87:0/2423091044 conn(0x7feec00d2230 msgr2=0x7feec00d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:52.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.953+0000 7feec714a640 1 -- 172.21.15.87:0/2423091044 shutdown_connections 2024-09-17T03:14:52.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.954+0000 7feec714a640 1 -- 172.21.15.87:0/2423091044 wait complete. 2024-09-17T03:14:52.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.955+0000 7feec714a640 1 Processor -- start 2024-09-17T03:14:52.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.955+0000 7feec714a640 1 -- start start 2024-09-17T03:14:52.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.957+0000 7feec714a640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feec00b76d0 0x7feec01621b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:52.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.957+0000 7feec714a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feec00cd690 0x7feec009f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:52.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.957+0000 7feec714a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feec00d0b40 0x7feec00b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:52.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.957+0000 7feec714a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7feec00b3040 con 0x7feec00cd690 2024-09-17T03:14:52.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.957+0000 7feec714a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7feec00b2ec0 con 0x7feec00d0b40 2024-09-17T03:14:52.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.957+0000 7feec714a640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7feec0168120 con 0x7feec00b76d0 2024-09-17T03:14:52.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.957+0000 7feeb7fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feec00cd690 0x7feec009f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:52.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.957+0000 7feeb7fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feec00cd690 0x7feec009f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34254/0 (socket says 172.21.15.87:34254) 2024-09-17T03:14:52.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.957+0000 7feeb7fff640 1 -- 172.21.15.87:0/673119200 learned_addr learned my addr 172.21.15.87:0/673119200 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:52.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.957+0000 7feec4ee6640 1 --2- 172.21.15.87:0/673119200 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feec00b76d0 0x7feec01621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:52.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.957+0000 7feec56e7640 1 --2- 172.21.15.87:0/673119200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feec00d0b40 0x7feec00b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:52.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.957+0000 7feeb7fff640 1 -- 172.21.15.87:0/673119200 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feec00b76d0 msgr2=0x7feec01621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:52.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.957+0000 7feeb7fff640 1 --2- 172.21.15.87:0/673119200 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feec00b76d0 0x7feec01621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:52.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.957+0000 7feeb7fff640 1 -- 172.21.15.87:0/673119200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feec00d0b40 msgr2=0x7feec00b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:52.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.957+0000 7feeb7fff640 1 --2- 172.21.15.87:0/673119200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feec00d0b40 0x7feec00b3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:52.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.957+0000 7feeb7fff640 1 -- 172.21.15.87:0/673119200 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec0163210 con 0x7feec00cd690 2024-09-17T03:14:52.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.957+0000 7feec4ee6640 1 --2- 172.21.15.87:0/673119200 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feec00b76d0 0x7feec01621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:14:52.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.957+0000 7feeb7fff640 1 --2- 172.21.15.87:0/673119200 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feec00cd690 0x7feec009f840 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7feeb000c3c0 tx=0x7feeb000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:52.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.957+0000 7feeb57fa640 1 -- 172.21.15.87:0/673119200 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb0017070 con 0x7feec00cd690 2024-09-17T03:14:52.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.957+0000 7feeb57fa640 1 -- 172.21.15.87:0/673119200 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feeb00035f0 con 0x7feec00cd690 2024-09-17T03:14:52.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.958+0000 7feec714a640 1 -- 172.21.15.87:0/673119200 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec0162c50 con 0x7feec00cd690 2024-09-17T03:14:52.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.958+0000 7feeb57fa640 1 -- 172.21.15.87:0/673119200 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb0003a80 con 0x7feec00cd690 2024-09-17T03:14:52.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.958+0000 7feec714a640 1 -- 172.21.15.87:0/673119200 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec01493f0 con 0x7feec00cd690 2024-09-17T03:14:52.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.959+0000 7feec714a640 1 -- 172.21.15.87:0/673119200 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec00d0f50 con 0x7feec00cd690 2024-09-17T03:14:52.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.959+0000 7feeb57fa640 1 -- 172.21.15.87:0/673119200 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7feeb001a070 con 0x7feec00cd690 2024-09-17T03:14:52.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.960+0000 7feeb57fa640 1 --2- 172.21.15.87:0/673119200 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7feeac078b90 0x7feeac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:52.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.960+0000 7feeb57fa640 1 -- 172.21.15.87:0/673119200 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7feeb0099c40 con 0x7feec00cd690 2024-09-17T03:14:52.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.960+0000 7feec4ee6640 1 --2- 172.21.15.87:0/673119200 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7feeac078b90 0x7feeac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:52.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.960+0000 7feec4ee6640 1 --2- 172.21.15.87:0/673119200 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7feeac078b90 0x7feeac07b0a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7feebc00a830 tx=0x7feebc008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:52.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:52.964+0000 7feeb57fa640 1 -- 172.21.15.87:0/673119200 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7feeb0062460 con 0x7feec00cd690 2024-09-17T03:14:53.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.137+0000 7feec714a640 1 -- 172.21.15.87:0/673119200 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7feec00cbfd0 con 0x7feec00cd690 2024-09-17T03:14:53.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.138+0000 7feeb57fa640 1 -- 172.21.15.87:0/673119200 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) ==== 84+0+65 (secure 0 0 0) 0x7feeb0061d60 con 0x7feec00cd690 2024-09-17T03:14:53.138 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:53.138 INFO:teuthology.orchestra.run.smithi087.stdout:[ 2024-09-17T03:14:53.138 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:14:53.138 INFO:teuthology.orchestra.run.smithi087.stdout: "poolnum": 1, 2024-09-17T03:14:53.138 INFO:teuthology.orchestra.run.smithi087.stdout: "poolname": ".mgr" 2024-09-17T03:14:53.138 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:14:53.138 INFO:teuthology.orchestra.run.smithi087.stdout:] 2024-09-17T03:14:53.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.140+0000 7feec714a640 1 -- 172.21.15.87:0/673119200 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7feeac078b90 msgr2=0x7feeac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:53.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.140+0000 7feec714a640 1 --2- 172.21.15.87:0/673119200 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7feeac078b90 0x7feeac07b0a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7feebc00a830 tx=0x7feebc008040 comp rx=0 tx=0).stop 2024-09-17T03:14:53.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.140+0000 7feec714a640 1 -- 172.21.15.87:0/673119200 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feec00cd690 msgr2=0x7feec009f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:53.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.140+0000 7feec714a640 1 --2- 172.21.15.87:0/673119200 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feec00cd690 0x7feec009f840 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7feeb000c3c0 tx=0x7feeb000c880 comp rx=0 tx=0).stop 2024-09-17T03:14:53.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.140+0000 7feec714a640 1 -- 172.21.15.87:0/673119200 shutdown_connections 2024-09-17T03:14:53.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.140+0000 7feec714a640 1 --2- 172.21.15.87:0/673119200 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7feeac078b90 0x7feeac07b0a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:53.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.140+0000 7feec714a640 1 --2- 172.21.15.87:0/673119200 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feec00d0b40 0x7feec00b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:53.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.140+0000 7feec714a640 1 --2- 172.21.15.87:0/673119200 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feec00cd690 0x7feec009f840 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:53.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.140+0000 7feec714a640 1 --2- 172.21.15.87:0/673119200 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feec00b76d0 0x7feec01621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:53.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.140+0000 7feec714a640 1 -- 172.21.15.87:0/673119200 >> 172.21.15.87:0/673119200 conn(0x7feec00d2230 msgr2=0x7feec0163b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:53.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.140+0000 7feec714a640 1 -- 172.21.15.87:0/673119200 shutdown_connections 2024-09-17T03:14:53.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.140+0000 7feec714a640 1 -- 172.21.15.87:0/673119200 wait complete. 2024-09-17T03:14:53.151 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T03:14:53.152 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T03:14:53.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.267+0000 7f04c6913640 1 -- 172.21.15.87:0/3827154530 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04c0070990 msgr2=0x7f04c00cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:53.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.267+0000 7f04c6913640 1 --2- 172.21.15.87:0/3827154530 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04c0070990 0x7f04c00cf070 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f04c006b360 tx=0x7f04b402a580 comp rx=0 tx=0).stop 2024-09-17T03:14:53.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.267+0000 7f04c6913640 1 -- 172.21.15.87:0/3827154530 shutdown_connections 2024-09-17T03:14:53.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.267+0000 7f04c6913640 1 --2- 172.21.15.87:0/3827154530 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04c0070990 0x7f04c00cf070 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:53.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.267+0000 7f04c6913640 1 --2- 172.21.15.87:0/3827154530 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f04c00cf5b0 0x7f04c00b4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:53.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.267+0000 7f04c6913640 1 --2- 172.21.15.87:0/3827154530 >> v2:172.21.15.149:3300/0 conn(0x7f04c015b940 0x7f04c0070450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:53.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.267+0000 7f04c6913640 1 -- 172.21.15.87:0/3827154530 >> 172.21.15.87:0/3827154530 conn(0x7f04c00b2cc0 msgr2=0x7f04c00b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:53.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.267+0000 7f04c6913640 1 -- 172.21.15.87:0/3827154530 shutdown_connections 2024-09-17T03:14:53.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.267+0000 7f04c6913640 1 -- 172.21.15.87:0/3827154530 wait complete. 2024-09-17T03:14:53.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.269+0000 7f04c6913640 1 Processor -- start 2024-09-17T03:14:53.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.269+0000 7f04c6913640 1 -- start start 2024-09-17T03:14:53.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.270+0000 7f04c6913640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04c0070990 0x7f04c014c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:53.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.270+0000 7f04c6913640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f04c00cf5b0 0x7f04c01492b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:53.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.270+0000 7f04c6913640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04c015b940 0x7f04c015e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:53.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.270+0000 7f04c6913640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f04c0070100 con 0x7f04c0070990 2024-09-17T03:14:53.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.270+0000 7f04c6913640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f04c006ff80 con 0x7f04c015b940 2024-09-17T03:14:53.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.270+0000 7f04c6913640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f04c00b52c0 con 0x7f04c00cf5b0 2024-09-17T03:14:53.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.270+0000 7f04bffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04c0070990 0x7f04c014c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:53.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.270+0000 7f04bffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04c0070990 0x7f04c014c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34270/0 (socket says 172.21.15.87:34270) 2024-09-17T03:14:53.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.270+0000 7f04bffff640 1 -- 172.21.15.87:0/996464617 learned_addr learned my addr 172.21.15.87:0/996464617 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:53.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.270+0000 7f04bf7fe640 1 --2- 172.21.15.87:0/996464617 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f04c00cf5b0 0x7f04c01492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:53.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.271+0000 7f04c4eb0640 1 --2- 172.21.15.87:0/996464617 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04c015b940 0x7f04c015e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:53.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.271+0000 7f04bffff640 1 -- 172.21.15.87:0/996464617 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f04c00cf5b0 msgr2=0x7f04c01492b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:53.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.271+0000 7f04bffff640 1 --2- 172.21.15.87:0/996464617 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f04c00cf5b0 0x7f04c01492b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:53.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.271+0000 7f04bffff640 1 -- 172.21.15.87:0/996464617 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04c015b940 msgr2=0x7f04c015e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:53.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.271+0000 7f04bffff640 1 --2- 172.21.15.87:0/996464617 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04c015b940 0x7f04c015e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:53.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.271+0000 7f04bffff640 1 -- 172.21.15.87:0/996464617 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c014cb30 con 0x7f04c0070990 2024-09-17T03:14:53.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.271+0000 7f04c4eb0640 1 --2- 172.21.15.87:0/996464617 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04c015b940 0x7f04c015e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:53.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.271+0000 7f04bffff640 1 --2- 172.21.15.87:0/996464617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04c0070990 0x7f04c014c5f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f04b000b410 tx=0x7f04b000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:53.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.271+0000 7f04bcff9640 1 -- 172.21.15.87:0/996464617 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0015020 con 0x7f04c0070990 2024-09-17T03:14:53.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.271+0000 7f04bcff9640 1 -- 172.21.15.87:0/996464617 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04b0011940 con 0x7f04c0070990 2024-09-17T03:14:53.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.271+0000 7f04c6913640 1 -- 172.21.15.87:0/996464617 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c015d210 con 0x7f04c0070990 2024-09-17T03:14:53.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.271+0000 7f04bcff9640 1 -- 172.21.15.87:0/996464617 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0011d90 con 0x7f04c0070990 2024-09-17T03:14:53.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.271+0000 7f04c6913640 1 -- 172.21.15.87:0/996464617 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c015d7d0 con 0x7f04c0070990 2024-09-17T03:14:53.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.272+0000 7f04c6913640 1 -- 172.21.15.87:0/996464617 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04c00b5880 con 0x7f04c0070990 2024-09-17T03:14:53.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.273+0000 7f04bcff9640 1 -- 172.21.15.87:0/996464617 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f04b0018020 con 0x7f04c0070990 2024-09-17T03:14:53.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.273+0000 7f04bcff9640 1 --2- 172.21.15.87:0/996464617 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f04ac078b90 0x7f04ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:53.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.273+0000 7f04bcff9640 1 -- 172.21.15.87:0/996464617 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f04b009a830 con 0x7f04c0070990 2024-09-17T03:14:53.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.274+0000 7f04bf7fe640 1 --2- 172.21.15.87:0/996464617 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f04ac078b90 0x7f04ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:53.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.274+0000 7f04bf7fe640 1 --2- 172.21.15.87:0/996464617 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f04ac078b90 0x7f04ac07b0a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f04b4033000 tx=0x7f04b4004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:53.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.278+0000 7f04bcff9640 1 -- 172.21.15.87:0/996464617 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f04b001f020 con 0x7f04c0070990 2024-09-17T03:14:53.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:53.454+0000 7f04c6913640 1 -- 172.21.15.87:0/996464617 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f04c00b0540 con 0x7f04c0070990 2024-09-17T03:14:54.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.307+0000 7f04bcff9640 1 -- 172.21.15.87:0/996464617 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) ==== 138+0+0 (secure 0 0 0) 0x7f04c014cb30 con 0x7f04c0070990 2024-09-17T03:14:54.306 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' created 2024-09-17T03:14:54.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.310+0000 7f04c6913640 1 -- 172.21.15.87:0/996464617 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f04ac078b90 msgr2=0x7f04ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:54.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.310+0000 7f04c6913640 1 --2- 172.21.15.87:0/996464617 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f04ac078b90 0x7f04ac07b0a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f04b4033000 tx=0x7f04b4004730 comp rx=0 tx=0).stop 2024-09-17T03:14:54.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.310+0000 7f04c6913640 1 -- 172.21.15.87:0/996464617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04c0070990 msgr2=0x7f04c014c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:54.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.310+0000 7f04c6913640 1 --2- 172.21.15.87:0/996464617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04c0070990 0x7f04c014c5f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f04b000b410 tx=0x7f04b000b8d0 comp rx=0 tx=0).stop 2024-09-17T03:14:54.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.310+0000 7f04c6913640 1 -- 172.21.15.87:0/996464617 shutdown_connections 2024-09-17T03:14:54.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.310+0000 7f04c6913640 1 --2- 172.21.15.87:0/996464617 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f04ac078b90 0x7f04ac07b0a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:54.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.310+0000 7f04c6913640 1 --2- 172.21.15.87:0/996464617 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04c015b940 0x7f04c015e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:54.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.310+0000 7f04c6913640 1 --2- 172.21.15.87:0/996464617 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f04c00cf5b0 0x7f04c01492b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:54.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.310+0000 7f04c6913640 1 --2- 172.21.15.87:0/996464617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04c0070990 0x7f04c014c5f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:54.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.310+0000 7f04c6913640 1 -- 172.21.15.87:0/996464617 >> 172.21.15.87:0/996464617 conn(0x7f04c00b2cc0 msgr2=0x7f04c0070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:54.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.310+0000 7f04c6913640 1 -- 172.21.15.87:0/996464617 shutdown_connections 2024-09-17T03:14:54.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.311+0000 7f04c6913640 1 -- 172.21.15.87:0/996464617 wait complete. 2024-09-17T03:14:54.324 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T03:14:54.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.450+0000 7f8816677640 1 -- 172.21.15.87:0/2039126008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88100b7de0 msgr2=0x7f8810160c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:54.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.450+0000 7f8816677640 1 --2- 172.21.15.87:0/2039126008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88100b7de0 0x7f8810160c80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f88100ce1a0 tx=0x7f880002a410 comp rx=0 tx=0).stop 2024-09-17T03:14:54.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.450+0000 7f8816677640 1 -- 172.21.15.87:0/2039126008 shutdown_connections 2024-09-17T03:14:54.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.450+0000 7f8816677640 1 --2- 172.21.15.87:0/2039126008 >> v2:172.21.15.149:3300/0 conn(0x7f88100b4430 0x7f88100d04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:54.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.450+0000 7f8816677640 1 --2- 172.21.15.87:0/2039126008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88100b7de0 0x7f8810160c80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:54.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.450+0000 7f8816677640 1 -- 172.21.15.87:0/2039126008 >> 172.21.15.87:0/2039126008 conn(0x7f8810069330 msgr2=0x7f88100ced20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:54.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.450+0000 7f8816677640 1 -- 172.21.15.87:0/2039126008 shutdown_connections 2024-09-17T03:14:54.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.450+0000 7f8816677640 1 -- 172.21.15.87:0/2039126008 wait complete. 2024-09-17T03:14:54.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.452+0000 7f8816677640 1 Processor -- start 2024-09-17T03:14:54.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.452+0000 7f8816677640 1 -- start start 2024-09-17T03:14:54.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.453+0000 7f8816677640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88100b7de0 0x7f88100c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:54.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.453+0000 7f8816677640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f881015c620 0x7f881014bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:54.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.453+0000 7f8816677640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88100b50d0 0x7f88100b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:54.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.454+0000 7f8816677640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f88100d0ed0 con 0x7f88100b7de0 2024-09-17T03:14:54.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.454+0000 7f8816677640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f88100d0d50 con 0x7f88100b50d0 2024-09-17T03:14:54.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.454+0000 7f8816677640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f88100d1050 con 0x7f881015c620 2024-09-17T03:14:54.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.454+0000 7f880ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88100b7de0 0x7f88100c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:54.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.454+0000 7f880ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88100b7de0 0x7f88100c8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34286/0 (socket says 172.21.15.87:34286) 2024-09-17T03:14:54.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.454+0000 7f880ffff640 1 -- 172.21.15.87:0/2156709105 learned_addr learned my addr 172.21.15.87:0/2156709105 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:54.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.454+0000 7f8814c14640 1 --2- 172.21.15.87:0/2156709105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88100b50d0 0x7f88100b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:54.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.454+0000 7f880f7fe640 1 --2- 172.21.15.87:0/2156709105 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f881015c620 0x7f881014bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:54.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.454+0000 7f880ffff640 1 -- 172.21.15.87:0/2156709105 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f881015c620 msgr2=0x7f881014bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:54.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.454+0000 7f880ffff640 1 --2- 172.21.15.87:0/2156709105 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f881015c620 0x7f881014bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:54.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.454+0000 7f880ffff640 1 -- 172.21.15.87:0/2156709105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88100b50d0 msgr2=0x7f88100b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:54.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.454+0000 7f880ffff640 1 --2- 172.21.15.87:0/2156709105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88100b50d0 0x7f88100b54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:54.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.454+0000 7f880ffff640 1 -- 172.21.15.87:0/2156709105 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88100b5a20 con 0x7f88100b7de0 2024-09-17T03:14:54.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.454+0000 7f880ffff640 1 --2- 172.21.15.87:0/2156709105 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88100b7de0 0x7f88100c8780 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f880002aa00 tx=0x7f8800033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:54.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.454+0000 7f880cff9640 1 -- 172.21.15.87:0/2156709105 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800005c90 con 0x7f88100b7de0 2024-09-17T03:14:54.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.454+0000 7f880cff9640 1 -- 172.21.15.87:0/2156709105 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8800031e50 con 0x7f88100b7de0 2024-09-17T03:14:54.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.454+0000 7f880cff9640 1 -- 172.21.15.87:0/2156709105 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8800005260 con 0x7f88100b7de0 2024-09-17T03:14:54.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.454+0000 7f8816677640 1 -- 172.21.15.87:0/2156709105 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88101f6390 con 0x7f88100b7de0 2024-09-17T03:14:54.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.454+0000 7f8816677640 1 -- 172.21.15.87:0/2156709105 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f881014c110 con 0x7f88100b7de0 2024-09-17T03:14:54.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.455+0000 7f8816677640 1 -- 172.21.15.87:0/2156709105 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88100d15e0 con 0x7f88100b7de0 2024-09-17T03:14:54.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.456+0000 7f880cff9640 1 -- 172.21.15.87:0/2156709105 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8800005400 con 0x7f88100b7de0 2024-09-17T03:14:54.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.457+0000 7f880cff9640 1 --2- 172.21.15.87:0/2156709105 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f87fc078bc0 0x7f87fc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:54.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.457+0000 7f880cff9640 1 -- 172.21.15.87:0/2156709105 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5227+0+0 (secure 0 0 0) 0x7f88000bbc30 con 0x7f88100b7de0 2024-09-17T03:14:54.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.457+0000 7f880f7fe640 1 --2- 172.21.15.87:0/2156709105 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f87fc078bc0 0x7f87fc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:54.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.457+0000 7f880f7fe640 1 --2- 172.21.15.87:0/2156709105 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f87fc078bc0 0x7f87fc07b0d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f881015b070 tx=0x7f8804009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:54.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.461+0000 7f880cff9640 1 -- 172.21.15.87:0/2156709105 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f88100b5a20 con 0x7f88100b7de0 2024-09-17T03:14:54.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:54.640+0000 7f8816677640 1 -- 172.21.15.87:0/2156709105 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f88101f6810 con 0x7f88100b7de0 2024-09-17T03:14:55.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.309+0000 7f880cff9640 1 -- 172.21.15.87:0/2156709105 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) ==== 170+0+0 (secure 0 0 0) 0x7f880003f030 con 0x7f88100b7de0 2024-09-17T03:14:55.308 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' created 2024-09-17T03:14:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.313+0000 7f8816677640 1 -- 172.21.15.87:0/2156709105 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f87fc078bc0 msgr2=0x7f87fc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.313+0000 7f8816677640 1 --2- 172.21.15.87:0/2156709105 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f87fc078bc0 0x7f87fc07b0d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f881015b070 tx=0x7f8804009040 comp rx=0 tx=0).stop 2024-09-17T03:14:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.313+0000 7f8816677640 1 -- 172.21.15.87:0/2156709105 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88100b7de0 msgr2=0x7f88100c8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.313+0000 7f8816677640 1 --2- 172.21.15.87:0/2156709105 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88100b7de0 0x7f88100c8780 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f880002aa00 tx=0x7f8800033c20 comp rx=0 tx=0).stop 2024-09-17T03:14:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.313+0000 7f8816677640 1 -- 172.21.15.87:0/2156709105 shutdown_connections 2024-09-17T03:14:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.313+0000 7f8816677640 1 --2- 172.21.15.87:0/2156709105 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f87fc078bc0 0x7f87fc07b0d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.313+0000 7f8816677640 1 --2- 172.21.15.87:0/2156709105 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f88100b50d0 0x7f88100b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.313+0000 7f8816677640 1 --2- 172.21.15.87:0/2156709105 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f881015c620 0x7f881014bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.313+0000 7f8816677640 1 --2- 172.21.15.87:0/2156709105 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88100b7de0 0x7f88100c8780 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.313+0000 7f8816677640 1 -- 172.21.15.87:0/2156709105 >> 172.21.15.87:0/2156709105 conn(0x7f8810069330 msgr2=0x7f881015b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:55.312 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.313+0000 7f8816677640 1 -- 172.21.15.87:0/2156709105 shutdown_connections 2024-09-17T03:14:55.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.313+0000 7f8816677640 1 -- 172.21.15.87:0/2156709105 wait complete. 2024-09-17T03:14:55.336 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T03:14:55.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.454+0000 7f608aab4640 1 -- 172.21.15.87:0/2776215780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f606c004d40 msgr2=0x7f606c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:55.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.454+0000 7f608aab4640 1 --2- 172.21.15.87:0/2776215780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f606c004d40 0x7f606c005150 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f60840c9300 tx=0x7f606c005830 comp rx=0 tx=0).stop 2024-09-17T03:14:55.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.454+0000 7f6089051640 1 -- 172.21.15.87:0/2776215780 reap_dead start 2024-09-17T03:14:55.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.454+0000 7f608aab4640 1 -- 172.21.15.87:0/2776215780 shutdown_connections 2024-09-17T03:14:55.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.454+0000 7f608aab4640 1 -- 172.21.15.87:0/2776215780 >> 172.21.15.87:0/2776215780 conn(0x7f6084069330 msgr2=0x7f60840ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:55.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.454+0000 7f608aab4640 1 -- 172.21.15.87:0/2776215780 shutdown_connections 2024-09-17T03:14:55.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.455+0000 7f608aab4640 1 -- 172.21.15.87:0/2776215780 wait complete. 2024-09-17T03:14:55.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.456+0000 7f608aab4640 1 Processor -- start 2024-09-17T03:14:55.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.456+0000 7f608aab4640 1 -- start start 2024-09-17T03:14:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.458+0000 7f608aab4640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f608415c530 0x7f608415ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.458+0000 7f608aab4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60841534e0 0x7f60840b7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.458+0000 7f608aab4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60840b77c0 0x7f60840ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.458+0000 7f608aab4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f60840cd370 con 0x7f60841534e0 2024-09-17T03:14:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.458+0000 7f608aab4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f60840cd1f0 con 0x7f60840b77c0 2024-09-17T03:14:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.458+0000 7f608aab4640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f608406f570 con 0x7f608415c530 2024-09-17T03:14:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.458+0000 7f607bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60841534e0 0x7f60840b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.458+0000 7f607bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60841534e0 0x7f60840b7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39886/0 (socket says 172.21.15.87:39886) 2024-09-17T03:14:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.458+0000 7f607bfff640 1 -- 172.21.15.87:0/1375674232 learned_addr learned my addr 172.21.15.87:0/1375674232 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.458+0000 7f6089051640 1 --2- 172.21.15.87:0/1375674232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60840b77c0 0x7f60840ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.458+0000 7f6088850640 1 --2- 172.21.15.87:0/1375674232 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f608415c530 0x7f608415ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.458+0000 7f607bfff640 1 -- 172.21.15.87:0/1375674232 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f608415c530 msgr2=0x7f608415ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.458+0000 7f607bfff640 1 --2- 172.21.15.87:0/1375674232 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f608415c530 0x7f608415ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.458+0000 7f607bfff640 1 -- 172.21.15.87:0/1375674232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60840b77c0 msgr2=0x7f60840ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:55.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.458+0000 7f607bfff640 1 --2- 172.21.15.87:0/1375674232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60840b77c0 0x7f60840ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:55.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.458+0000 7f607bfff640 1 -- 172.21.15.87:0/1375674232 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60840b7bd0 con 0x7f60841534e0 2024-09-17T03:14:55.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.458+0000 7f6089051640 1 --2- 172.21.15.87:0/1375674232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60840b77c0 0x7f60840ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:55.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.458+0000 7f6088850640 1 --2- 172.21.15.87:0/1375674232 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f608415c530 0x7f608415ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:55.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.458+0000 7f607bfff640 1 --2- 172.21.15.87:0/1375674232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60841534e0 0x7f60840b7280 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6074011ae0 tx=0x7f6074009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:55.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.458+0000 7f60797fa640 1 -- 172.21.15.87:0/1375674232 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607401e070 con 0x7f60841534e0 2024-09-17T03:14:55.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.458+0000 7f60797fa640 1 -- 172.21.15.87:0/1375674232 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6074004460 con 0x7f60841534e0 2024-09-17T03:14:55.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.458+0000 7f608aab4640 1 -- 172.21.15.87:0/1375674232 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60841f9a20 con 0x7f60841534e0 2024-09-17T03:14:55.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.458+0000 7f60797fa640 1 -- 172.21.15.87:0/1375674232 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607400fc60 con 0x7f60841534e0 2024-09-17T03:14:55.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.459+0000 7f608aab4640 1 -- 172.21.15.87:0/1375674232 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60840b1590 con 0x7f60841534e0 2024-09-17T03:14:55.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.459+0000 7f608aab4640 1 -- 172.21.15.87:0/1375674232 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f608406e890 con 0x7f60841534e0 2024-09-17T03:14:55.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.460+0000 7f60797fa640 1 -- 172.21.15.87:0/1375674232 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6074021070 con 0x7f60841534e0 2024-09-17T03:14:55.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.461+0000 7f60797fa640 1 --2- 172.21.15.87:0/1375674232 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6070078b90 0x7f607007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:55.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.461+0000 7f60797fa640 1 -- 172.21.15.87:0/1375674232 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5609+0+0 (secure 0 0 0) 0x7f607409b6e0 con 0x7f60841534e0 2024-09-17T03:14:55.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.461+0000 7f6088850640 1 --2- 172.21.15.87:0/1375674232 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6070078b90 0x7f607007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:55.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.462+0000 7f6088850640 1 --2- 172.21.15.87:0/1375674232 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6070078b90 0x7f607007b0a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6080009cd0 tx=0x7f6080005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:55.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.466+0000 7f60797fa640 1 -- 172.21.15.87:0/1375674232 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f60840b7bd0 con 0x7f60841534e0 2024-09-17T03:14:55.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:55.674+0000 7f608aab4640 1 -- 172.21.15.87:0/1375674232 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f60840cca50 con 0x7f60841534e0 2024-09-17T03:14:56.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.320+0000 7f60797fa640 1 -- 172.21.15.87:0/1375674232 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) ==== 172+0+0 (secure 0 0 0) 0x7f6074063630 con 0x7f60841534e0 2024-09-17T03:14:56.319 INFO:teuthology.orchestra.run.smithi087.stderr:new fs with metadata pool 4 and data pool 5 2024-09-17T03:14:56.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.324+0000 7f608aab4640 1 -- 172.21.15.87:0/1375674232 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6070078b90 msgr2=0x7f607007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:56.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.324+0000 7f608aab4640 1 --2- 172.21.15.87:0/1375674232 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6070078b90 0x7f607007b0a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6080009cd0 tx=0x7f6080005f90 comp rx=0 tx=0).stop 2024-09-17T03:14:56.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.324+0000 7f608aab4640 1 -- 172.21.15.87:0/1375674232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60841534e0 msgr2=0x7f60840b7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:56.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.324+0000 7f608aab4640 1 --2- 172.21.15.87:0/1375674232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60841534e0 0x7f60840b7280 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6074011ae0 tx=0x7f6074009e40 comp rx=0 tx=0).stop 2024-09-17T03:14:56.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.326+0000 7f608aab4640 1 -- 172.21.15.87:0/1375674232 shutdown_connections 2024-09-17T03:14:56.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.326+0000 7f608aab4640 1 --2- 172.21.15.87:0/1375674232 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6070078b90 0x7f607007b0a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:56.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.326+0000 7f608aab4640 1 --2- 172.21.15.87:0/1375674232 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f60840b77c0 0x7f60840ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:56.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.326+0000 7f608aab4640 1 --2- 172.21.15.87:0/1375674232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60841534e0 0x7f60840b7280 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:56.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.326+0000 7f608aab4640 1 --2- 172.21.15.87:0/1375674232 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f608415c530 0x7f608415ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:56.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.326+0000 7f608aab4640 1 -- 172.21.15.87:0/1375674232 >> 172.21.15.87:0/1375674232 conn(0x7f6084069330 msgr2=0x7f60840b2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:56.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.329+0000 7f608aab4640 1 -- 172.21.15.87:0/1375674232 shutdown_connections 2024-09-17T03:14:56.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.330+0000 7f608aab4640 1 -- 172.21.15.87:0/1375674232 wait complete. 2024-09-17T03:14:56.354 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:14:56.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.469+0000 7fd47b420640 1 -- 172.21.15.87:0/398754427 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd45c004800 msgr2=0x7fd45c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:56.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.469+0000 7fd47b420640 1 --2- 172.21.15.87:0/398754427 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd45c004800 0x7fd45c004c10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd46800c2e0 tx=0x7fd45c005830 comp rx=0 tx=0).stop 2024-09-17T03:14:56.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.469+0000 7fd4799bd640 1 -- 172.21.15.87:0/398754427 reap_dead start 2024-09-17T03:14:56.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.469+0000 7fd47b420640 1 -- 172.21.15.87:0/398754427 shutdown_connections 2024-09-17T03:14:56.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.469+0000 7fd47b420640 1 -- 172.21.15.87:0/398754427 >> 172.21.15.87:0/398754427 conn(0x7fd474069360 msgr2=0x7fd4740ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:56.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.469+0000 7fd47b420640 1 -- 172.21.15.87:0/398754427 shutdown_connections 2024-09-17T03:14:56.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.469+0000 7fd47b420640 1 -- 172.21.15.87:0/398754427 wait complete. 2024-09-17T03:14:56.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.471+0000 7fd47b420640 1 Processor -- start 2024-09-17T03:14:56.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.471+0000 7fd47b420640 1 -- start start 2024-09-17T03:14:56.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.472+0000 7fd47b420640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd45c004800 0x7fd474158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:56.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.472+0000 7fd47b420640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4741493d0 0x7fd4740cb410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:56.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.472+0000 7fd47b420640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4740cb950 0x7fd4741f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:56.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.472+0000 7fd47b420640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd4740d0870 con 0x7fd4741493d0 2024-09-17T03:14:56.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.472+0000 7fd47b420640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd4740d09f0 con 0x7fd4740cb950 2024-09-17T03:14:56.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.472+0000 7fd47b420640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd4740d0b70 con 0x7fd45c004800 2024-09-17T03:14:56.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.472+0000 7fd4789bb640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4741493d0 0x7fd4740cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:56.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.473+0000 7fd4789bb640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4741493d0 0x7fd4740cb410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39910/0 (socket says 172.21.15.87:39910) 2024-09-17T03:14:56.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.473+0000 7fd4789bb640 1 -- 172.21.15.87:0/1147670106 learned_addr learned my addr 172.21.15.87:0/1147670106 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:56.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.473+0000 7fd4799bd640 1 --2- 172.21.15.87:0/1147670106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4740cb950 0x7fd4741f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:56.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.473+0000 7fd4791bc640 1 --2- 172.21.15.87:0/1147670106 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd45c004800 0x7fd474158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:56.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.473+0000 7fd4789bb640 1 -- 172.21.15.87:0/1147670106 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd45c004800 msgr2=0x7fd474158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:56.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.473+0000 7fd4789bb640 1 --2- 172.21.15.87:0/1147670106 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd45c004800 0x7fd474158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:56.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.473+0000 7fd4789bb640 1 -- 172.21.15.87:0/1147670106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4740cb950 msgr2=0x7fd4741f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:56.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.473+0000 7fd4789bb640 1 --2- 172.21.15.87:0/1147670106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4740cb950 0x7fd4741f34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:56.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.473+0000 7fd4789bb640 1 -- 172.21.15.87:0/1147670106 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4740ca650 con 0x7fd4741493d0 2024-09-17T03:14:56.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.473+0000 7fd4789bb640 1 --2- 172.21.15.87:0/1147670106 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4741493d0 0x7fd4740cb410 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd46800c400 tx=0x7fd468006ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:56.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.473+0000 7fd471ffb640 1 -- 172.21.15.87:0/1147670106 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468031040 con 0x7fd4741493d0 2024-09-17T03:14:56.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.473+0000 7fd471ffb640 1 -- 172.21.15.87:0/1147670106 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4680065e0 con 0x7fd4741493d0 2024-09-17T03:14:56.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.473+0000 7fd471ffb640 1 -- 172.21.15.87:0/1147670106 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd468003a40 con 0x7fd4741493d0 2024-09-17T03:14:56.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.473+0000 7fd47b420640 1 -- 172.21.15.87:0/1147670106 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4740ca930 con 0x7fd4741493d0 2024-09-17T03:14:56.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.473+0000 7fd47b420640 1 -- 172.21.15.87:0/1147670106 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4740cbd60 con 0x7fd4741493d0 2024-09-17T03:14:56.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.474+0000 7fd47b420640 1 -- 172.21.15.87:0/1147670106 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4741f77e0 con 0x7fd4741493d0 2024-09-17T03:14:56.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.475+0000 7fd471ffb640 1 -- 172.21.15.87:0/1147670106 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd468034070 con 0x7fd4741493d0 2024-09-17T03:14:56.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.476+0000 7fd471ffb640 1 --2- 172.21.15.87:0/1147670106 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd460078b90 0x7fd46007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:56.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.476+0000 7fd471ffb640 1 -- 172.21.15.87:0/1147670106 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd4680bad10 con 0x7fd4741493d0 2024-09-17T03:14:56.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.476+0000 7fd4791bc640 1 --2- 172.21.15.87:0/1147670106 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd460078b90 0x7fd46007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:56.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.477+0000 7fd4791bc640 1 --2- 172.21.15.87:0/1147670106 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd460078b90 0x7fd46007b0a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd464005fd0 tx=0x7fd464009650 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:56.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.480+0000 7fd471ffb640 1 -- 172.21.15.87:0/1147670106 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd468082b90 con 0x7fd4741493d0 2024-09-17T03:14:56.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.655+0000 7fd47b420640 1 -- 172.21.15.87:0/1147670106 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4740ce500 con 0x7fd4741493d0 2024-09-17T03:14:56.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.656+0000 7fd471ffb640 1 -- 172.21.15.87:0/1147670106 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16706 (secure 0 0 0) 0x7fd4740ce500 con 0x7fd4741493d0 2024-09-17T03:14:56.655 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:56.655 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":33,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:14:55.675659+0000","last_up_change":"2024-09-17T03:14:01.833605+0000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:14:03.433404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:14:53.459693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T03:14:54.645011+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6825","nonce":2430409749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6827","nonce":2430409749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6831","nonce":2430409749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6829","nonce":2430409749}]},"public_addr":"172.21.15.87:6825/2430409749","cluster_addr":"172.21.15.87:6827/2430409749","heartbeat_back_addr":"172.21.15.87:6831/2430409749","heartbeat_front_addr":"172.21.15.87:6829/2430409749","state":["exists","up"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6809","nonce":3467605435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6811","nonce":3467605435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6815","nonce":3467605435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6813","nonce":3467605435}]},"public_addr":"172.21.15.87:6809/3467605435","cluster_addr":"172.21.15.87:6811/3467605435","heartbeat_back_addr":"172.21.15.87:6815/3467605435","heartbeat_front_addr":"172.21.15.87:6813/3467605435","state":["exists","up"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6801","nonce":1450834766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6803","nonce":1450834766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6807","nonce":1450834766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6805","nonce":1450834766}]},"public_addr":"172.21.15.87:6801/1450834766","cluster_addr":"172.21.15.87:6803/1450834766","heartbeat_back_addr":"172.21.15.87:6807/1450834766","heartbeat_front_addr":"172.21.15.87:6805/1450834766","state":["exists","up"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6817","nonce":4253161216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6819","nonce":4253161216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6823","nonce":4253161216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6821","nonce":4253161216}]},"public_addr":"172.21.15.87:6817/4253161216","cluster_addr":"172.21.15.87:6819/4253161216","heartbeat_back_addr":"172.21.15.87:6823/4253161216","heartbeat_front_addr":"172.21.15.87:6821/4253161216","state":["exists","up"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6803","nonce":2414907089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6805","nonce":2414907089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6809","nonce":2414907089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6807","nonce":2414907089}]},"public_addr":"172.21.15.149:6803/2414907089","cluster_addr":"172.21.15.149:6805/2414907089","heartbeat_back_addr":"172.21.15.149:6809/2414907089","heartbeat_front_addr":"172.21.15.149:6807/2414907089","state":["exists","up"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6827","nonce":2182999530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6829","nonce":2182999530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6833","nonce":2182999530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6831","nonce":2182999530}]},"public_addr":"172.21.15.149:6827/2182999530","cluster_addr":"172.21.15.149:6829/2182999530","heartbeat_back_addr":"172.21.15.149:6833/2182999530","heartbeat_front_addr":"172.21.15.149:6831/2182999530","state":["exists","up"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6811","nonce":4289519146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6813","nonce":4289519146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6817","nonce":4289519146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6815","nonce":4289519146}]},"public_addr":"172.21.15.149:6811/4289519146","cluster_addr":"172.21.15.149:6813/4289519146","heartbeat_back_addr":"172.21.15.149:6817/4289519146","heartbeat_front_addr":"172.21.15.149:6815/4289519146","state":["exists","up"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6819","nonce":2771010477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6821","nonce":2771010477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6825","nonce":2771010477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6823","nonce":2771010477}]},"public_addr":"172.21.15.149:6819/2771010477","cluster_addr":"172.21.15.149:6821/2771010477","heartbeat_back_addr":"172.21.15.149:6825/2771010477","heartbeat_front_addr":"172.21.15.149:6823/2771010477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.313297+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.642861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.654781+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.302606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.385791+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.915266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.121365+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.394432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:14:56.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.658+0000 7fd47b420640 1 -- 172.21.15.87:0/1147670106 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd460078b90 msgr2=0x7fd46007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:56.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.658+0000 7fd47b420640 1 --2- 172.21.15.87:0/1147670106 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd460078b90 0x7fd46007b0a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd464005fd0 tx=0x7fd464009650 comp rx=0 tx=0).stop 2024-09-17T03:14:56.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.658+0000 7fd47b420640 1 -- 172.21.15.87:0/1147670106 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4741493d0 msgr2=0x7fd4740cb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:56.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.658+0000 7fd47b420640 1 --2- 172.21.15.87:0/1147670106 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4741493d0 0x7fd4740cb410 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd46800c400 tx=0x7fd468006ce0 comp rx=0 tx=0).stop 2024-09-17T03:14:56.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.658+0000 7fd47b420640 1 -- 172.21.15.87:0/1147670106 shutdown_connections 2024-09-17T03:14:56.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.658+0000 7fd47b420640 1 --2- 172.21.15.87:0/1147670106 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd460078b90 0x7fd46007b0a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:56.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.658+0000 7fd47b420640 1 --2- 172.21.15.87:0/1147670106 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd4740cb950 0x7fd4741f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:56.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.658+0000 7fd47b420640 1 --2- 172.21.15.87:0/1147670106 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4741493d0 0x7fd4740cb410 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:56.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.658+0000 7fd47b420640 1 --2- 172.21.15.87:0/1147670106 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd45c004800 0x7fd474158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:56.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.658+0000 7fd47b420640 1 -- 172.21.15.87:0/1147670106 >> 172.21.15.87:0/1147670106 conn(0x7fd474069360 msgr2=0x7fd4740b6c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:56.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.659+0000 7fd47b420640 1 -- 172.21.15.87:0/1147670106 shutdown_connections 2024-09-17T03:14:56.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.659+0000 7fd47b420640 1 -- 172.21.15.87:0/1147670106 wait complete. 2024-09-17T03:14:56.669 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:14:56.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.782+0000 7fe10ffff640 1 -- 172.21.15.87:0/3758295509 <== mon.1 v2:172.21.15.149:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1080047a0 con 0x7fe1180b3d60 2024-09-17T03:14:56.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.783+0000 7fe11f2c4640 1 -- 172.21.15.87:0/3758295509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1180b3d60 msgr2=0x7fe1180b4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:56.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.783+0000 7fe11f2c4640 1 --2- 172.21.15.87:0/3758295509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1180b3d60 0x7fe1180b4170 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe108005fd0 tx=0x7fe10802a540 comp rx=0 tx=0).stop 2024-09-17T03:14:56.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.783+0000 7fe11f2c4640 1 -- 172.21.15.87:0/3758295509 shutdown_connections 2024-09-17T03:14:56.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.783+0000 7fe11f2c4640 1 --2- 172.21.15.87:0/3758295509 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1180b3d60 0x7fe1180b4170 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:56.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.783+0000 7fe11f2c4640 1 --2- 172.21.15.87:0/3758295509 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1180b7f10 0x7fe11815c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:56.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.783+0000 7fe11f2c4640 1 -- 172.21.15.87:0/3758295509 >> 172.21.15.87:0/3758295509 conn(0x7fe118069330 msgr2=0x7fe1180ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:56.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.783+0000 7fe11f2c4640 1 -- 172.21.15.87:0/3758295509 shutdown_connections 2024-09-17T03:14:56.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.783+0000 7fe11f2c4640 1 -- 172.21.15.87:0/3758295509 wait complete. 2024-09-17T03:14:56.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.785+0000 7fe11f2c4640 1 Processor -- start 2024-09-17T03:14:56.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.785+0000 7fe11f2c4640 1 -- start start 2024-09-17T03:14:56.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.786+0000 7fe11f2c4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1180710e0 0x7fe1181f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:56.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.786+0000 7fe11f2c4640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1180b3d60 0x7fe1180b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:56.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.786+0000 7fe11f2c4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1180b7f10 0x7fe1180ab360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:56.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.786+0000 7fe11f2c4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe1180b7b10 con 0x7fe1180710e0 2024-09-17T03:14:56.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.786+0000 7fe11f2c4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe1180d1c30 con 0x7fe1180b7f10 2024-09-17T03:14:56.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.786+0000 7fe11f2c4640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe1180b7c90 con 0x7fe1180b3d60 2024-09-17T03:14:56.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.786+0000 7fe11d060640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1180710e0 0x7fe1181f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:56.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.786+0000 7fe11d060640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1180710e0 0x7fe1181f3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39924/0 (socket says 172.21.15.87:39924) 2024-09-17T03:14:56.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.786+0000 7fe11d060640 1 -- 172.21.15.87:0/1964577170 learned_addr learned my addr 172.21.15.87:0/1964577170 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:56.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.786+0000 7fe11c85f640 1 --2- 172.21.15.87:0/1964577170 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1180b3d60 0x7fe1180b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:56.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.786+0000 7fe11d861640 1 --2- 172.21.15.87:0/1964577170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1180b7f10 0x7fe1180ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:56.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.787+0000 7fe11d060640 1 -- 172.21.15.87:0/1964577170 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1180b3d60 msgr2=0x7fe1180b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:56.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.787+0000 7fe11d060640 1 --2- 172.21.15.87:0/1964577170 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1180b3d60 0x7fe1180b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:56.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.787+0000 7fe11d060640 1 -- 172.21.15.87:0/1964577170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1180b7f10 msgr2=0x7fe1180ab360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:56.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.787+0000 7fe11d060640 1 --2- 172.21.15.87:0/1964577170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1180b7f10 0x7fe1180ab360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:56.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.787+0000 7fe11d060640 1 -- 172.21.15.87:0/1964577170 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1180a9a60 con 0x7fe1180710e0 2024-09-17T03:14:56.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.787+0000 7fe11c85f640 1 --2- 172.21.15.87:0/1964577170 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1180b3d60 0x7fe1180b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:56.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.787+0000 7fe11d060640 1 --2- 172.21.15.87:0/1964577170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1180710e0 0x7fe1181f3d30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe11400e9f0 tx=0x7fe11400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:56.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.787+0000 7fe10dffb640 1 -- 172.21.15.87:0/1964577170 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11400cd80 con 0x7fe1180710e0 2024-09-17T03:14:56.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.787+0000 7fe10dffb640 1 -- 172.21.15.87:0/1964577170 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe114004020 con 0x7fe1180710e0 2024-09-17T03:14:56.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.787+0000 7fe11f2c4640 1 -- 172.21.15.87:0/1964577170 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1180a9d40 con 0x7fe1180710e0 2024-09-17T03:14:56.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.787+0000 7fe10dffb640 1 -- 172.21.15.87:0/1964577170 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe114004470 con 0x7fe1180710e0 2024-09-17T03:14:56.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.787+0000 7fe11f2c4640 1 -- 172.21.15.87:0/1964577170 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1180aa300 con 0x7fe1180710e0 2024-09-17T03:14:56.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.788+0000 7fe11f2c4640 1 -- 172.21.15.87:0/1964577170 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1180ce320 con 0x7fe1180710e0 2024-09-17T03:14:56.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.789+0000 7fe10dffb640 1 -- 172.21.15.87:0/1964577170 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe1140035f0 con 0x7fe1180710e0 2024-09-17T03:14:56.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.789+0000 7fe10dffb640 1 --2- 172.21.15.87:0/1964577170 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe0fc078bc0 0x7fe0fc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:56.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.789+0000 7fe10dffb640 1 -- 172.21.15.87:0/1964577170 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fe11409b8c0 con 0x7fe1180710e0 2024-09-17T03:14:56.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.789+0000 7fe11c85f640 1 --2- 172.21.15.87:0/1964577170 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe0fc078bc0 0x7fe0fc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:56.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.790+0000 7fe11c85f640 1 --2- 172.21.15.87:0/1964577170 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe0fc078bc0 0x7fe0fc07b0d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe108033000 tx=0x7fe108004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:56.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.794+0000 7fe10dffb640 1 -- 172.21.15.87:0/1964577170 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe1180ce320 con 0x7fe1180710e0 2024-09-17T03:14:56.967 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.968+0000 7fe11f2c4640 1 -- 172.21.15.87:0/1964577170 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe118071de0 con 0x7fe1180710e0 2024-09-17T03:14:56.968 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.969+0000 7fe10dffb640 1 -- 172.21.15.87:0/1964577170 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16706 (secure 0 0 0) 0x7fe114063c80 con 0x7fe1180710e0 2024-09-17T03:14:56.968 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:56.968 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":33,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:14:55.675659+0000","last_up_change":"2024-09-17T03:14:01.833605+0000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:14:03.433404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:14:53.459693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T03:14:54.645011+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6825","nonce":2430409749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6827","nonce":2430409749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6831","nonce":2430409749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6829","nonce":2430409749}]},"public_addr":"172.21.15.87:6825/2430409749","cluster_addr":"172.21.15.87:6827/2430409749","heartbeat_back_addr":"172.21.15.87:6831/2430409749","heartbeat_front_addr":"172.21.15.87:6829/2430409749","state":["exists","up"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6809","nonce":3467605435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6811","nonce":3467605435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6815","nonce":3467605435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6813","nonce":3467605435}]},"public_addr":"172.21.15.87:6809/3467605435","cluster_addr":"172.21.15.87:6811/3467605435","heartbeat_back_addr":"172.21.15.87:6815/3467605435","heartbeat_front_addr":"172.21.15.87:6813/3467605435","state":["exists","up"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6801","nonce":1450834766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6803","nonce":1450834766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6807","nonce":1450834766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6805","nonce":1450834766}]},"public_addr":"172.21.15.87:6801/1450834766","cluster_addr":"172.21.15.87:6803/1450834766","heartbeat_back_addr":"172.21.15.87:6807/1450834766","heartbeat_front_addr":"172.21.15.87:6805/1450834766","state":["exists","up"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6817","nonce":4253161216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6819","nonce":4253161216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6823","nonce":4253161216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6821","nonce":4253161216}]},"public_addr":"172.21.15.87:6817/4253161216","cluster_addr":"172.21.15.87:6819/4253161216","heartbeat_back_addr":"172.21.15.87:6823/4253161216","heartbeat_front_addr":"172.21.15.87:6821/4253161216","state":["exists","up"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6803","nonce":2414907089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6805","nonce":2414907089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6809","nonce":2414907089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6807","nonce":2414907089}]},"public_addr":"172.21.15.149:6803/2414907089","cluster_addr":"172.21.15.149:6805/2414907089","heartbeat_back_addr":"172.21.15.149:6809/2414907089","heartbeat_front_addr":"172.21.15.149:6807/2414907089","state":["exists","up"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6827","nonce":2182999530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6829","nonce":2182999530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6833","nonce":2182999530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6831","nonce":2182999530}]},"public_addr":"172.21.15.149:6827/2182999530","cluster_addr":"172.21.15.149:6829/2182999530","heartbeat_back_addr":"172.21.15.149:6833/2182999530","heartbeat_front_addr":"172.21.15.149:6831/2182999530","state":["exists","up"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6811","nonce":4289519146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6813","nonce":4289519146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6817","nonce":4289519146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6815","nonce":4289519146}]},"public_addr":"172.21.15.149:6811/4289519146","cluster_addr":"172.21.15.149:6813/4289519146","heartbeat_back_addr":"172.21.15.149:6817/4289519146","heartbeat_front_addr":"172.21.15.149:6815/4289519146","state":["exists","up"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6819","nonce":2771010477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6821","nonce":2771010477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6825","nonce":2771010477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6823","nonce":2771010477}]},"public_addr":"172.21.15.149:6819/2771010477","cluster_addr":"172.21.15.149:6821/2771010477","heartbeat_back_addr":"172.21.15.149:6825/2771010477","heartbeat_front_addr":"172.21.15.149:6823/2771010477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.313297+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.642861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.654781+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.302606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.385791+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.915266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.121365+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.394432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:14:56.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.971+0000 7fe11f2c4640 1 -- 172.21.15.87:0/1964577170 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe0fc078bc0 msgr2=0x7fe0fc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:56.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.971+0000 7fe11f2c4640 1 --2- 172.21.15.87:0/1964577170 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe0fc078bc0 0x7fe0fc07b0d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe108033000 tx=0x7fe108004730 comp rx=0 tx=0).stop 2024-09-17T03:14:56.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.971+0000 7fe11f2c4640 1 -- 172.21.15.87:0/1964577170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1180710e0 msgr2=0x7fe1181f3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:56.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.971+0000 7fe11f2c4640 1 --2- 172.21.15.87:0/1964577170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1180710e0 0x7fe1181f3d30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe11400e9f0 tx=0x7fe11400eeb0 comp rx=0 tx=0).stop 2024-09-17T03:14:56.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.971+0000 7fe11f2c4640 1 -- 172.21.15.87:0/1964577170 shutdown_connections 2024-09-17T03:14:56.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.971+0000 7fe11f2c4640 1 --2- 172.21.15.87:0/1964577170 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe0fc078bc0 0x7fe0fc07b0d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:56.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.971+0000 7fe11f2c4640 1 --2- 172.21.15.87:0/1964577170 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe1180b7f10 0x7fe1180ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:56.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.971+0000 7fe11f2c4640 1 --2- 172.21.15.87:0/1964577170 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe1180b3d60 0x7fe1180b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:56.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.971+0000 7fe11f2c4640 1 --2- 172.21.15.87:0/1964577170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1180710e0 0x7fe1181f3d30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:56.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.971+0000 7fe11f2c4640 1 -- 172.21.15.87:0/1964577170 >> 172.21.15.87:0/1964577170 conn(0x7fe118069330 msgr2=0x7fe11815ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:56.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.971+0000 7fe11f2c4640 1 -- 172.21.15.87:0/1964577170 shutdown_connections 2024-09-17T03:14:56.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:56.971+0000 7fe11f2c4640 1 -- 172.21.15.87:0/1964577170 wait complete. 2024-09-17T03:14:56.982 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T03:14:57.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.097+0000 7fbeb5fc6640 1 -- 172.21.15.87:0/394903243 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe98004800 msgr2=0x7fbe98004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:57.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.097+0000 7fbeb5fc6640 1 --2- 172.21.15.87:0/394903243 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe98004800 0x7fbe98004c10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbea400bbe0 tx=0x7fbe98005830 comp rx=0 tx=0).stop 2024-09-17T03:14:57.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.097+0000 7fbeb5fc6640 1 -- 172.21.15.87:0/394903243 shutdown_connections 2024-09-17T03:14:57.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.097+0000 7fbeb5fc6640 1 --2- 172.21.15.87:0/394903243 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe98004800 0x7fbe98004c10 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:57.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.097+0000 7fbeb5fc6640 1 -- 172.21.15.87:0/394903243 >> 172.21.15.87:0/394903243 conn(0x7fbeb00cdc80 msgr2=0x7fbeb00ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:57.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.097+0000 7fbeb5fc6640 1 -- 172.21.15.87:0/394903243 shutdown_connections 2024-09-17T03:14:57.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.097+0000 7fbeb5fc6640 1 -- 172.21.15.87:0/394903243 wait complete. 2024-09-17T03:14:57.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.099+0000 7fbeb5fc6640 1 Processor -- start 2024-09-17T03:14:57.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.099+0000 7fbeb5fc6640 1 -- start start 2024-09-17T03:14:57.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.101+0000 7fbeb5fc6640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbe98004800 0x7fbeb00ca480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:57.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.101+0000 7fbeb5fc6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbeb00ca9c0 0x7fbeb00b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:57.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.101+0000 7fbeb5fc6640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbeb00b4650 0x7fbeb01f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:57.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.101+0000 7fbeb5fc6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb00cd3f0 con 0x7fbeb00ca9c0 2024-09-17T03:14:57.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.101+0000 7fbeb5fc6640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbeb0072220 con 0x7fbeb00b4650 2024-09-17T03:14:57.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.101+0000 7fbeb5fc6640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbeb00cd570 con 0x7fbe98004800 2024-09-17T03:14:57.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.101+0000 7fbeaeffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbeb00ca9c0 0x7fbeb00b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:57.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.101+0000 7fbeaeffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbeb00ca9c0 0x7fbeb00b4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39944/0 (socket says 172.21.15.87:39944) 2024-09-17T03:14:57.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.101+0000 7fbeaeffd640 1 -- 172.21.15.87:0/2877648090 learned_addr learned my addr 172.21.15.87:0/2877648090 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:57.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.101+0000 7fbeaf7fe640 1 --2- 172.21.15.87:0/2877648090 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbe98004800 0x7fbeb00ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:57.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.101+0000 7fbeaffff640 1 --2- 172.21.15.87:0/2877648090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbeb00b4650 0x7fbeb01f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:57.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.101+0000 7fbeaeffd640 1 -- 172.21.15.87:0/2877648090 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbe98004800 msgr2=0x7fbeb00ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:57.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.101+0000 7fbeaeffd640 1 --2- 172.21.15.87:0/2877648090 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbe98004800 0x7fbeb00ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:57.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.101+0000 7fbeaeffd640 1 -- 172.21.15.87:0/2877648090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbeb00b4650 msgr2=0x7fbeb01f24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:57.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.101+0000 7fbeaeffd640 1 --2- 172.21.15.87:0/2877648090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbeb00b4650 0x7fbeb01f24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:57.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.101+0000 7fbeaeffd640 1 -- 172.21.15.87:0/2877648090 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb00cadd0 con 0x7fbeb00ca9c0 2024-09-17T03:14:57.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.101+0000 7fbeaffff640 1 --2- 172.21.15.87:0/2877648090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbeb00b4650 0x7fbeb01f24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:57.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.101+0000 7fbeaeffd640 1 --2- 172.21.15.87:0/2877648090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbeb00ca9c0 0x7fbeb00b4110 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbea4034040 tx=0x7fbea40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:57.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.101+0000 7fbe97fff640 1 -- 172.21.15.87:0/2877648090 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea403b5e0 con 0x7fbeb00ca9c0 2024-09-17T03:14:57.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.101+0000 7fbe97fff640 1 -- 172.21.15.87:0/2877648090 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbea403f070 con 0x7fbeb00ca9c0 2024-09-17T03:14:57.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.101+0000 7fbeb5fc6640 1 -- 172.21.15.87:0/2877648090 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeb00c82b0 con 0x7fbeb00ca9c0 2024-09-17T03:14:57.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.101+0000 7fbe97fff640 1 -- 172.21.15.87:0/2877648090 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea403bb10 con 0x7fbeb00ca9c0 2024-09-17T03:14:57.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.102+0000 7fbeb5fc6640 1 -- 172.21.15.87:0/2877648090 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeb00c8a00 con 0x7fbeb00ca9c0 2024-09-17T03:14:57.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.103+0000 7fbeb5fc6640 1 -- 172.21.15.87:0/2877648090 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeb00d0050 con 0x7fbeb00ca9c0 2024-09-17T03:14:57.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.103+0000 7fbe97fff640 1 -- 172.21.15.87:0/2877648090 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbea4038070 con 0x7fbeb00ca9c0 2024-09-17T03:14:57.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.104+0000 7fbe97fff640 1 --2- 172.21.15.87:0/2877648090 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbe9c078bc0 0x7fbe9c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:57.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.104+0000 7fbe97fff640 1 -- 172.21.15.87:0/2877648090 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fbea40bdd40 con 0x7fbeb00ca9c0 2024-09-17T03:14:57.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.104+0000 7fbeaf7fe640 1 --2- 172.21.15.87:0/2877648090 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbe9c078bc0 0x7fbe9c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:57.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.105+0000 7fbeaf7fe640 1 --2- 172.21.15.87:0/2877648090 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbe9c078bc0 0x7fbe9c07b0d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbea0001330 tx=0x7fbea0006040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:57.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.108+0000 7fbe97fff640 1 -- 172.21.15.87:0/2877648090 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fbeb00cadd0 con 0x7fbeb00ca9c0 2024-09-17T03:14:57.317 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.318+0000 7fbeb5fc6640 1 -- 172.21.15.87:0/2877648090 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbeb01599c0 con 0x7fbeb00ca9c0 2024-09-17T03:14:57.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.332+0000 7fbe97fff640 1 -- 172.21.15.87:0/2877648090 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v12) ==== 118+0+0 (secure 0 0 0) 0x7fbea4086210 con 0x7fbeb00ca9c0 2024-09-17T03:14:57.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.334+0000 7fbeb5fc6640 1 -- 172.21.15.87:0/2877648090 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbe9c078bc0 msgr2=0x7fbe9c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:57.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.334+0000 7fbeb5fc6640 1 --2- 172.21.15.87:0/2877648090 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbe9c078bc0 0x7fbe9c07b0d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbea0001330 tx=0x7fbea0006040 comp rx=0 tx=0).stop 2024-09-17T03:14:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.334+0000 7fbeb5fc6640 1 -- 172.21.15.87:0/2877648090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbeb00ca9c0 msgr2=0x7fbeb00b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.334+0000 7fbeb5fc6640 1 --2- 172.21.15.87:0/2877648090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbeb00ca9c0 0x7fbeb00b4110 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fbea4034040 tx=0x7fbea40039c0 comp rx=0 tx=0).stop 2024-09-17T03:14:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.334+0000 7fbeb5fc6640 1 -- 172.21.15.87:0/2877648090 shutdown_connections 2024-09-17T03:14:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.334+0000 7fbeb5fc6640 1 --2- 172.21.15.87:0/2877648090 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbe9c078bc0 0x7fbe9c07b0d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.334+0000 7fbeb5fc6640 1 --2- 172.21.15.87:0/2877648090 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbeb00b4650 0x7fbeb01f24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.334+0000 7fbeb5fc6640 1 --2- 172.21.15.87:0/2877648090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbeb00ca9c0 0x7fbeb00b4110 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.334+0000 7fbeb5fc6640 1 --2- 172.21.15.87:0/2877648090 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbe98004800 0x7fbeb00ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.334+0000 7fbeb5fc6640 1 -- 172.21.15.87:0/2877648090 >> 172.21.15.87:0/2877648090 conn(0x7fbeb00cdc80 msgr2=0x7fbeb00b1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.335+0000 7fbeb5fc6640 1 -- 172.21.15.87:0/2877648090 shutdown_connections 2024-09-17T03:14:57.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.335+0000 7fbeb5fc6640 1 -- 172.21.15.87:0/2877648090 wait complete. 2024-09-17T03:14:57.345 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T03:14:57.345 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T03:14:57.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.462+0000 7fd658751640 1 -- 172.21.15.87:0/27429945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd63c005150 msgr2=0x7fd63c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:57.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.462+0000 7fd658751640 1 --2- 172.21.15.87:0/27429945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd63c005150 0x7fd63c00b9c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd648009ed0 tx=0x7fd63c005830 comp rx=0 tx=0).stop 2024-09-17T03:14:57.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.462+0000 7fd656cee640 1 -- 172.21.15.87:0/27429945 reap_dead start 2024-09-17T03:14:57.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.462+0000 7fd658751640 1 -- 172.21.15.87:0/27429945 shutdown_connections 2024-09-17T03:14:57.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.462+0000 7fd658751640 1 -- 172.21.15.87:0/27429945 >> 172.21.15.87:0/27429945 conn(0x7fd650069330 msgr2=0x7fd6500ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:57.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.462+0000 7fd658751640 1 -- 172.21.15.87:0/27429945 shutdown_connections 2024-09-17T03:14:57.461 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.462+0000 7fd658751640 1 -- 172.21.15.87:0/27429945 wait complete. 2024-09-17T03:14:57.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.464+0000 7fd658751640 1 Processor -- start 2024-09-17T03:14:57.462 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.464+0000 7fd658751640 1 -- start start 2024-09-17T03:14:57.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.465+0000 7fd658751640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd63c005150 0x7fd6501596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:57.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.465+0000 7fd658751640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd6501f65b0 0x7fd6500b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:57.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.465+0000 7fd658751640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd650156a60 0x7fd6501f3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:57.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.465+0000 7fd658751640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd6500b4620 con 0x7fd63c005150 2024-09-17T03:14:57.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.465+0000 7fd658751640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd6500cd520 con 0x7fd650156a60 2024-09-17T03:14:57.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.465+0000 7fd658751640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd650160560 con 0x7fd6501f65b0 2024-09-17T03:14:57.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.465+0000 7fd6564ed640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd63c005150 0x7fd6501596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:57.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.465+0000 7fd6564ed640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd63c005150 0x7fd6501596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39962/0 (socket says 172.21.15.87:39962) 2024-09-17T03:14:57.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.465+0000 7fd6564ed640 1 -- 172.21.15.87:0/1972047983 learned_addr learned my addr 172.21.15.87:0/1972047983 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:57.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.465+0000 7fd656cee640 1 --2- 172.21.15.87:0/1972047983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd650156a60 0x7fd6501f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:57.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.465+0000 7fd655cec640 1 --2- 172.21.15.87:0/1972047983 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd6501f65b0 0x7fd6500b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:57.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.465+0000 7fd6564ed640 1 -- 172.21.15.87:0/1972047983 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd6501f65b0 msgr2=0x7fd6500b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:57.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.465+0000 7fd6564ed640 1 --2- 172.21.15.87:0/1972047983 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd6501f65b0 0x7fd6500b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:57.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.465+0000 7fd6564ed640 1 -- 172.21.15.87:0/1972047983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd650156a60 msgr2=0x7fd6501f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:57.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.465+0000 7fd6564ed640 1 --2- 172.21.15.87:0/1972047983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd650156a60 0x7fd6501f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:57.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.465+0000 7fd6564ed640 1 -- 172.21.15.87:0/1972047983 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6501f2f70 con 0x7fd63c005150 2024-09-17T03:14:57.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.466+0000 7fd656cee640 1 --2- 172.21.15.87:0/1972047983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd650156a60 0x7fd6501f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:57.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.466+0000 7fd655cec640 1 --2- 172.21.15.87:0/1972047983 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd6501f65b0 0x7fd6500b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:57.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.466+0000 7fd6564ed640 1 --2- 172.21.15.87:0/1972047983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd63c005150 0x7fd6501596c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd64c011ae0 tx=0x7fd64c009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:57.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.466+0000 7fd646ffd640 1 -- 172.21.15.87:0/1972047983 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c01e070 con 0x7fd63c005150 2024-09-17T03:14:57.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.466+0000 7fd646ffd640 1 -- 172.21.15.87:0/1972047983 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd64c00fc70 con 0x7fd63c005150 2024-09-17T03:14:57.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.466+0000 7fd658751640 1 -- 172.21.15.87:0/1972047983 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6501f6d90 con 0x7fd63c005150 2024-09-17T03:14:57.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.466+0000 7fd646ffd640 1 -- 172.21.15.87:0/1972047983 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c019410 con 0x7fd63c005150 2024-09-17T03:14:57.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.466+0000 7fd658751640 1 -- 172.21.15.87:0/1972047983 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6501f7350 con 0x7fd63c005150 2024-09-17T03:14:57.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.467+0000 7fd658751640 1 -- 172.21.15.87:0/1972047983 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6500ccda0 con 0x7fd63c005150 2024-09-17T03:14:57.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.467+0000 7fd646ffd640 1 -- 172.21.15.87:0/1972047983 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd64c021070 con 0x7fd63c005150 2024-09-17T03:14:57.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.468+0000 7fd646ffd640 1 --2- 172.21.15.87:0/1972047983 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd634078b90 0x7fd63407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:57.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.468+0000 7fd646ffd640 1 -- 172.21.15.87:0/1972047983 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fd64c09b5c0 con 0x7fd63c005150 2024-09-17T03:14:57.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.468+0000 7fd655cec640 1 --2- 172.21.15.87:0/1972047983 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd634078b90 0x7fd63407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:57.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.469+0000 7fd655cec640 1 --2- 172.21.15.87:0/1972047983 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd634078b90 0x7fd63407b0a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd640001a30 tx=0x7fd6400011f0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:57.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.473+0000 7fd646ffd640 1 -- 172.21.15.87:0/1972047983 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd64c0633b0 con 0x7fd63c005150 2024-09-17T03:14:57.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:57.682+0000 7fd658751640 1 -- 172.21.15.87:0/1972047983 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd6500cbc90 con 0x7fd63c005150 2024-09-17T03:14:58.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.341+0000 7fd646ffd640 1 -- 172.21.15.87:0/1972047983 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v13) ==== 122+0+0 (secure 0 0 0) 0x7fd64c067de0 con 0x7fd63c005150 2024-09-17T03:14:58.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.344+0000 7fd658751640 1 -- 172.21.15.87:0/1972047983 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd634078b90 msgr2=0x7fd63407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:58.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.344+0000 7fd658751640 1 --2- 172.21.15.87:0/1972047983 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd634078b90 0x7fd63407b0a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd640001a30 tx=0x7fd6400011f0 comp rx=0 tx=0).stop 2024-09-17T03:14:58.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.344+0000 7fd658751640 1 -- 172.21.15.87:0/1972047983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd63c005150 msgr2=0x7fd6501596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:58.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.344+0000 7fd658751640 1 --2- 172.21.15.87:0/1972047983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd63c005150 0x7fd6501596c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd64c011ae0 tx=0x7fd64c009eb0 comp rx=0 tx=0).stop 2024-09-17T03:14:58.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.344+0000 7fd658751640 1 -- 172.21.15.87:0/1972047983 shutdown_connections 2024-09-17T03:14:58.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.344+0000 7fd658751640 1 --2- 172.21.15.87:0/1972047983 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd634078b90 0x7fd63407b0a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:58.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.344+0000 7fd658751640 1 --2- 172.21.15.87:0/1972047983 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd650156a60 0x7fd6501f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:58.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.344+0000 7fd658751640 1 --2- 172.21.15.87:0/1972047983 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd6501f65b0 0x7fd6500b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:58.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.344+0000 7fd658751640 1 --2- 172.21.15.87:0/1972047983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd63c005150 0x7fd6501596c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:58.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.344+0000 7fd658751640 1 -- 172.21.15.87:0/1972047983 >> 172.21.15.87:0/1972047983 conn(0x7fd650069330 msgr2=0x7fd65015bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:58.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.344+0000 7fd658751640 1 -- 172.21.15.87:0/1972047983 shutdown_connections 2024-09-17T03:14:58.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.344+0000 7fd658751640 1 -- 172.21.15.87:0/1972047983 wait complete. 2024-09-17T03:14:58.355 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T03:14:58.356 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:14:58.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.472+0000 7f8ca8f5e640 1 -- 172.21.15.87:0/1140450159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ca415ecf0 msgr2=0x7f8ca415f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:58.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.472+0000 7f8ca8f5e640 1 --2- 172.21.15.87:0/1140450159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ca415ecf0 0x7f8ca415f100 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009830 tx=0x7f8c8c02a5c0 comp rx=0 tx=0).stop 2024-09-17T03:14:58.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.472+0000 7f8ca8f5e640 1 -- 172.21.15.87:0/1140450159 shutdown_connections 2024-09-17T03:14:58.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.472+0000 7f8ca8f5e640 1 --2- 172.21.15.87:0/1140450159 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ca415ecf0 0x7f8ca415f100 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:58.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.472+0000 7f8ca8f5e640 1 -- 172.21.15.87:0/1140450159 >> 172.21.15.87:0/1140450159 conn(0x7f8ca40d4470 msgr2=0x7f8ca40d4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:58.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.472+0000 7f8ca8f5e640 1 -- 172.21.15.87:0/1140450159 shutdown_connections 2024-09-17T03:14:58.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.472+0000 7f8ca8f5e640 1 -- 172.21.15.87:0/1140450159 wait complete. 2024-09-17T03:14:58.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.474+0000 7f8ca8f5e640 1 Processor -- start 2024-09-17T03:14:58.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.475+0000 7f8ca8f5e640 1 -- start start 2024-09-17T03:14:58.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.476+0000 7f8ca8f5e640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8ca40b5ef0 0x7f8ca414b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:58.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.476+0000 7f8ca8f5e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ca415c2d0 0x7f8ca40cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:58.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.476+0000 7f8ca8f5e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ca415ecf0 0x7f8ca40cda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:58.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.476+0000 7f8ca8f5e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca40d2fc0 con 0x7f8ca415c2d0 2024-09-17T03:14:58.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.476+0000 7f8ca8f5e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca40d2e40 con 0x7f8ca415ecf0 2024-09-17T03:14:58.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.476+0000 7f8ca8f5e640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8ca40d3140 con 0x7f8ca40b5ef0 2024-09-17T03:14:58.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.476+0000 7f8c99d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ca415c2d0 0x7f8ca40cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:58.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.476+0000 7f8c99d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ca415c2d0 0x7f8ca40cb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39974/0 (socket says 172.21.15.87:39974) 2024-09-17T03:14:58.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.476+0000 7f8c99d9b640 1 -- 172.21.15.87:0/1227836470 learned_addr learned my addr 172.21.15.87:0/1227836470 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:58.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.476+0000 7f8ca259c640 1 --2- 172.21.15.87:0/1227836470 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8ca40b5ef0 0x7f8ca414b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:58.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.476+0000 7f8ca2d9d640 1 --2- 172.21.15.87:0/1227836470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ca415ecf0 0x7f8ca40cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:58.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.476+0000 7f8c99d9b640 1 -- 172.21.15.87:0/1227836470 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8ca40b5ef0 msgr2=0x7f8ca414b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:58.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.476+0000 7f8c99d9b640 1 --2- 172.21.15.87:0/1227836470 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8ca40b5ef0 0x7f8ca414b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:58.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.476+0000 7f8c99d9b640 1 -- 172.21.15.87:0/1227836470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ca415ecf0 msgr2=0x7f8ca40cda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:58.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.476+0000 7f8c99d9b640 1 --2- 172.21.15.87:0/1227836470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ca415ecf0 0x7f8ca40cda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:58.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.476+0000 7f8c99d9b640 1 -- 172.21.15.87:0/1227836470 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca40cc4c0 con 0x7f8ca415c2d0 2024-09-17T03:14:58.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.476+0000 7f8ca259c640 1 --2- 172.21.15.87:0/1227836470 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8ca40b5ef0 0x7f8ca414b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:58.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.477+0000 7f8c99d9b640 1 --2- 172.21.15.87:0/1227836470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ca415c2d0 0x7f8ca40cb4b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009950 tx=0x7f8c8c033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:58.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.477+0000 7f8c9b7fe640 1 -- 172.21.15.87:0/1227836470 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c031b60 con 0x7f8ca415c2d0 2024-09-17T03:14:58.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.477+0000 7f8c9b7fe640 1 -- 172.21.15.87:0/1227836470 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c8c031df0 con 0x7f8ca415c2d0 2024-09-17T03:14:58.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.477+0000 7f8ca8f5e640 1 -- 172.21.15.87:0/1227836470 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca40cc1e0 con 0x7f8ca415c2d0 2024-09-17T03:14:58.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.477+0000 7f8c9b7fe640 1 -- 172.21.15.87:0/1227836470 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c005280 con 0x7f8ca415c2d0 2024-09-17T03:14:58.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.477+0000 7f8ca8f5e640 1 -- 172.21.15.87:0/1227836470 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca40cd210 con 0x7f8ca415c2d0 2024-09-17T03:14:58.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.478+0000 7f8ca8f5e640 1 -- 172.21.15.87:0/1227836470 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca40cbf50 con 0x7f8ca415c2d0 2024-09-17T03:14:58.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.478+0000 7f8c9b7fe640 1 -- 172.21.15.87:0/1227836470 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8c8c005420 con 0x7f8ca415c2d0 2024-09-17T03:14:58.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.479+0000 7f8c9b7fe640 1 --2- 172.21.15.87:0/1227836470 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8c84078b90 0x7f8c8407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:58.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.479+0000 7f8c9b7fe640 1 -- 172.21.15.87:0/1227836470 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f8c8c0bbe40 con 0x7f8ca415c2d0 2024-09-17T03:14:58.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.479+0000 7f8ca259c640 1 --2- 172.21.15.87:0/1227836470 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8c84078b90 0x7f8c8407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:58.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.480+0000 7f8ca259c640 1 --2- 172.21.15.87:0/1227836470 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8c84078b90 0x7f8c8407b0a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8c94007fd0 tx=0x7f8c94009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:58.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.483+0000 7f8c9b7fe640 1 -- 172.21.15.87:0/1227836470 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8c8c03f030 con 0x7f8ca415c2d0 2024-09-17T03:14:58.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.693+0000 7f8ca8f5e640 1 -- 172.21.15.87:0/1227836470 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ca40d35c0 con 0x7f8ca415c2d0 2024-09-17T03:14:58.692 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.693+0000 7f8c9b7fe640 1 -- 172.21.15.87:0/1227836470 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+5226 (secure 0 0 0) 0x7f8c8c031770 con 0x7f8ca415c2d0 2024-09-17T03:14:58.692 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:58.692 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":13,"btime":"2024-09-17T03:14:58:337521+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6836/1081794135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1081794135},{"type":"v1","addr":"172.21.15.149:6836","nonce":1081794135}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2152665509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6835","nonce":2152665509},{"type":"v1","addr":"172.21.15.149:6837","nonce":2152665509}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/704821751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":704821751},{"type":"v1","addr":"172.21.15.87:6835","nonce":704821751}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:14:55.675616+0000","modified":"2024-09-17T03:14:57.683823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5004},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5004":{"gid":5004,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6834/1243349323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":1243349323},{"type":"v1","addr":"172.21.15.87:6834","nonce":1243349323}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5004,"qdb_cluster":[5004]},"id":2}]} 2024-09-17T03:14:58.692 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 13 2024-09-17T03:14:58.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.695+0000 7f8ca8f5e640 1 -- 172.21.15.87:0/1227836470 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8c84078b90 msgr2=0x7f8c8407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:58.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.695+0000 7f8ca8f5e640 1 --2- 172.21.15.87:0/1227836470 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8c84078b90 0x7f8c8407b0a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8c94007fd0 tx=0x7f8c94009040 comp rx=0 tx=0).stop 2024-09-17T03:14:58.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.695+0000 7f8ca8f5e640 1 -- 172.21.15.87:0/1227836470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ca415c2d0 msgr2=0x7f8ca40cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:58.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.695+0000 7f8ca8f5e640 1 --2- 172.21.15.87:0/1227836470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ca415c2d0 0x7f8ca40cb4b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c009950 tx=0x7f8c8c033c10 comp rx=0 tx=0).stop 2024-09-17T03:14:58.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.695+0000 7f8ca8f5e640 1 -- 172.21.15.87:0/1227836470 shutdown_connections 2024-09-17T03:14:58.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.695+0000 7f8ca8f5e640 1 --2- 172.21.15.87:0/1227836470 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8c84078b90 0x7f8c8407b0a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:58.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.695+0000 7f8ca8f5e640 1 --2- 172.21.15.87:0/1227836470 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8ca415ecf0 0x7f8ca40cda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:58.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.695+0000 7f8ca8f5e640 1 --2- 172.21.15.87:0/1227836470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ca415c2d0 0x7f8ca40cb4b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:58.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.695+0000 7f8ca8f5e640 1 --2- 172.21.15.87:0/1227836470 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8ca40b5ef0 0x7f8ca414b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:58.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.695+0000 7f8ca8f5e640 1 -- 172.21.15.87:0/1227836470 >> 172.21.15.87:0/1227836470 conn(0x7f8ca40d4470 msgr2=0x7f8ca40d18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:58.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.695+0000 7f8ca8f5e640 1 -- 172.21.15.87:0/1227836470 shutdown_connections 2024-09-17T03:14:58.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.695+0000 7f8ca8f5e640 1 -- 172.21.15.87:0/1227836470 wait complete. 2024-09-17T03:14:58.705 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:14:58.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.819+0000 7f444bc3f640 1 -- 172.21.15.87:0/601428217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f444415acf0 msgr2=0x7f444415f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:58.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.819+0000 7f444bc3f640 1 --2- 172.21.15.87:0/601428217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f444415acf0 0x7f444415f000 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4434009830 tx=0x7f443402a530 comp rx=0 tx=0).stop 2024-09-17T03:14:58.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.819+0000 7f444bc3f640 1 -- 172.21.15.87:0/601428217 shutdown_connections 2024-09-17T03:14:58.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.819+0000 7f444bc3f640 1 --2- 172.21.15.87:0/601428217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f444415acf0 0x7f444415f000 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:58.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.819+0000 7f444bc3f640 1 -- 172.21.15.87:0/601428217 >> 172.21.15.87:0/601428217 conn(0x7f444406ee70 msgr2=0x7f444406f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:58.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.819+0000 7f444bc3f640 1 -- 172.21.15.87:0/601428217 shutdown_connections 2024-09-17T03:14:58.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.819+0000 7f444bc3f640 1 -- 172.21.15.87:0/601428217 wait complete. 2024-09-17T03:14:58.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.821+0000 7f444bc3f640 1 Processor -- start 2024-09-17T03:14:58.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.821+0000 7f444bc3f640 1 -- start start 2024-09-17T03:14:58.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.822+0000 7f444bc3f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44440af680 0x7f4444159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:58.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.822+0000 7f444bc3f640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44440cb440 0x7f44441f9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:58.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.822+0000 7f444bc3f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f444415acf0 0x7f4444154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:58.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.822+0000 7f444bc3f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f44440cc0e0 con 0x7f44440af680 2024-09-17T03:14:58.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.822+0000 7f444bc3f640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f44440cbf60 con 0x7f444415acf0 2024-09-17T03:14:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.822+0000 7f444bc3f640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f44440cc260 con 0x7f44440cb440 2024-09-17T03:14:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.822+0000 7f44499db640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44440af680 0x7f4444159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.822+0000 7f44499db640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44440af680 0x7f4444159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39982/0 (socket says 172.21.15.87:39982) 2024-09-17T03:14:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.822+0000 7f44499db640 1 -- 172.21.15.87:0/3042240003 learned_addr learned my addr 172.21.15.87:0/3042240003 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.822+0000 7f44491da640 1 --2- 172.21.15.87:0/3042240003 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44440cb440 0x7f44441f9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.822+0000 7f444a1dc640 1 --2- 172.21.15.87:0/3042240003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f444415acf0 0x7f4444154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.822+0000 7f44499db640 1 -- 172.21.15.87:0/3042240003 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44440cb440 msgr2=0x7f44441f9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.822+0000 7f44499db640 1 --2- 172.21.15.87:0/3042240003 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44440cb440 0x7f44441f9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.823+0000 7f44499db640 1 -- 172.21.15.87:0/3042240003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f444415acf0 msgr2=0x7f4444154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.823+0000 7f44499db640 1 --2- 172.21.15.87:0/3042240003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f444415acf0 0x7f4444154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.823+0000 7f44499db640 1 -- 172.21.15.87:0/3042240003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4444157620 con 0x7f44440af680 2024-09-17T03:14:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.823+0000 7f44491da640 1 --2- 172.21.15.87:0/3042240003 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44440cb440 0x7f44441f9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:14:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.823+0000 7f44499db640 1 --2- 172.21.15.87:0/3042240003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44440af680 0x7f4444159ef0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f444000c430 tx=0x7f444000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:58.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.823+0000 7f443a7fc640 1 -- 172.21.15.87:0/3042240003 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4440017070 con 0x7f44440af680 2024-09-17T03:14:58.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.823+0000 7f443a7fc640 1 -- 172.21.15.87:0/3042240003 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44400053c0 con 0x7f44440af680 2024-09-17T03:14:58.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.823+0000 7f444bc3f640 1 -- 172.21.15.87:0/3042240003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4444157900 con 0x7f44440af680 2024-09-17T03:14:58.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.823+0000 7f443a7fc640 1 -- 172.21.15.87:0/3042240003 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4440003850 con 0x7f44440af680 2024-09-17T03:14:58.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.823+0000 7f444bc3f640 1 -- 172.21.15.87:0/3042240003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44441f9160 con 0x7f44440af680 2024-09-17T03:14:58.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.824+0000 7f444bc3f640 1 -- 172.21.15.87:0/3042240003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44440cb850 con 0x7f44440af680 2024-09-17T03:14:58.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.825+0000 7f443a7fc640 1 -- 172.21.15.87:0/3042240003 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f444001a070 con 0x7f44440af680 2024-09-17T03:14:58.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.825+0000 7f443a7fc640 1 --2- 172.21.15.87:0/3042240003 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4428078bc0 0x7f442807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:58.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.825+0000 7f443a7fc640 1 -- 172.21.15.87:0/3042240003 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f4440099f80 con 0x7f44440af680 2024-09-17T03:14:58.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.825+0000 7f44491da640 1 --2- 172.21.15.87:0/3042240003 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4428078bc0 0x7f442807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:58.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.826+0000 7f44491da640 1 --2- 172.21.15.87:0/3042240003 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4428078bc0 0x7f442807b0d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4434034000 tx=0x7f4434004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:58.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:58.830+0000 7f443a7fc640 1 -- 172.21.15.87:0/3042240003 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4440061e00 con 0x7f44440af680 2024-09-17T03:14:59.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.005+0000 7f444bc3f640 1 -- 172.21.15.87:0/3042240003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4444076350 con 0x7f44440af680 2024-09-17T03:14:59.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.006+0000 7f443a7fc640 1 -- 172.21.15.87:0/3042240003 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16693 (secure 0 0 0) 0x7f4440066830 con 0x7f44440af680 2024-09-17T03:14:59.005 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:59.005 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":34,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:14:57.332130+0000","last_up_change":"2024-09-17T03:14:01.833605+0000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:14:03.433404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:14:53.459693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T03:14:54.645011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6825","nonce":2430409749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6827","nonce":2430409749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6831","nonce":2430409749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6829","nonce":2430409749}]},"public_addr":"172.21.15.87:6825/2430409749","cluster_addr":"172.21.15.87:6827/2430409749","heartbeat_back_addr":"172.21.15.87:6831/2430409749","heartbeat_front_addr":"172.21.15.87:6829/2430409749","state":["exists","up"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6809","nonce":3467605435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6811","nonce":3467605435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6815","nonce":3467605435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6813","nonce":3467605435}]},"public_addr":"172.21.15.87:6809/3467605435","cluster_addr":"172.21.15.87:6811/3467605435","heartbeat_back_addr":"172.21.15.87:6815/3467605435","heartbeat_front_addr":"172.21.15.87:6813/3467605435","state":["exists","up"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6801","nonce":1450834766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6803","nonce":1450834766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6807","nonce":1450834766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6805","nonce":1450834766}]},"public_addr":"172.21.15.87:6801/1450834766","cluster_addr":"172.21.15.87:6803/1450834766","heartbeat_back_addr":"172.21.15.87:6807/1450834766","heartbeat_front_addr":"172.21.15.87:6805/1450834766","state":["exists","up"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6817","nonce":4253161216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6819","nonce":4253161216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6823","nonce":4253161216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6821","nonce":4253161216}]},"public_addr":"172.21.15.87:6817/4253161216","cluster_addr":"172.21.15.87:6819/4253161216","heartbeat_back_addr":"172.21.15.87:6823/4253161216","heartbeat_front_addr":"172.21.15.87:6821/4253161216","state":["exists","up"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6803","nonce":2414907089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6805","nonce":2414907089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6809","nonce":2414907089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6807","nonce":2414907089}]},"public_addr":"172.21.15.149:6803/2414907089","cluster_addr":"172.21.15.149:6805/2414907089","heartbeat_back_addr":"172.21.15.149:6809/2414907089","heartbeat_front_addr":"172.21.15.149:6807/2414907089","state":["exists","up"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6827","nonce":2182999530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6829","nonce":2182999530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6833","nonce":2182999530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6831","nonce":2182999530}]},"public_addr":"172.21.15.149:6827/2182999530","cluster_addr":"172.21.15.149:6829/2182999530","heartbeat_back_addr":"172.21.15.149:6833/2182999530","heartbeat_front_addr":"172.21.15.149:6831/2182999530","state":["exists","up"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6811","nonce":4289519146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6813","nonce":4289519146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6817","nonce":4289519146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6815","nonce":4289519146}]},"public_addr":"172.21.15.149:6811/4289519146","cluster_addr":"172.21.15.149:6813/4289519146","heartbeat_back_addr":"172.21.15.149:6817/4289519146","heartbeat_front_addr":"172.21.15.149:6815/4289519146","state":["exists","up"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6819","nonce":2771010477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6821","nonce":2771010477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6825","nonce":2771010477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6823","nonce":2771010477}]},"public_addr":"172.21.15.149:6819/2771010477","cluster_addr":"172.21.15.149:6821/2771010477","heartbeat_back_addr":"172.21.15.149:6825/2771010477","heartbeat_front_addr":"172.21.15.149:6823/2771010477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.313297+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.642861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.654781+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.302606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.385791+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.915266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.121365+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.394432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:14:59.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.008+0000 7f444bc3f640 1 -- 172.21.15.87:0/3042240003 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4428078bc0 msgr2=0x7f442807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:59.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.008+0000 7f444bc3f640 1 --2- 172.21.15.87:0/3042240003 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4428078bc0 0x7f442807b0d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4434034000 tx=0x7f4434004730 comp rx=0 tx=0).stop 2024-09-17T03:14:59.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.008+0000 7f444bc3f640 1 -- 172.21.15.87:0/3042240003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44440af680 msgr2=0x7f4444159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:59.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.008+0000 7f444bc3f640 1 --2- 172.21.15.87:0/3042240003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44440af680 0x7f4444159ef0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f444000c430 tx=0x7f444000c8f0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.008+0000 7f444bc3f640 1 -- 172.21.15.87:0/3042240003 shutdown_connections 2024-09-17T03:14:59.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.008+0000 7f444bc3f640 1 --2- 172.21.15.87:0/3042240003 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4428078bc0 0x7f442807b0d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.008+0000 7f444bc3f640 1 --2- 172.21.15.87:0/3042240003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f444415acf0 0x7f4444154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.008+0000 7f444bc3f640 1 --2- 172.21.15.87:0/3042240003 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f44440cb440 0x7f44441f9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.008+0000 7f444bc3f640 1 --2- 172.21.15.87:0/3042240003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44440af680 0x7f4444159ef0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.008+0000 7f444bc3f640 1 -- 172.21.15.87:0/3042240003 >> 172.21.15.87:0/3042240003 conn(0x7f444406ee70 msgr2=0x7f44440afc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:59.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.008+0000 7f444bc3f640 1 -- 172.21.15.87:0/3042240003 shutdown_connections 2024-09-17T03:14:59.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.008+0000 7f444bc3f640 1 -- 172.21.15.87:0/3042240003 wait complete. 2024-09-17T03:14:59.018 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T03:14:59.019 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:14:59.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.132+0000 7fd6634f4640 1 -- 172.21.15.87:0/1958066685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd65c0b4470 msgr2=0x7fd65c15c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:59.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.132+0000 7fd6634f4640 1 --2- 172.21.15.87:0/1958066685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd65c0b4470 0x7fd65c15c5f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd64c009860 tx=0x7fd64c02a4b0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.132+0000 7fd6634f4640 1 -- 172.21.15.87:0/1958066685 shutdown_connections 2024-09-17T03:14:59.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.132+0000 7fd6634f4640 1 --2- 172.21.15.87:0/1958066685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd65c0b4470 0x7fd65c15c5f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.132+0000 7fd6634f4640 1 -- 172.21.15.87:0/1958066685 >> 172.21.15.87:0/1958066685 conn(0x7fd65c069330 msgr2=0x7fd65c0ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:59.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.132+0000 7fd6634f4640 1 -- 172.21.15.87:0/1958066685 shutdown_connections 2024-09-17T03:14:59.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.132+0000 7fd6634f4640 1 -- 172.21.15.87:0/1958066685 wait complete. 2024-09-17T03:14:59.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.134+0000 7fd6634f4640 1 Processor -- start 2024-09-17T03:14:59.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.134+0000 7fd6634f4640 1 -- start start 2024-09-17T03:14:59.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.135+0000 7fd6634f4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd65c0b7f10 0x7fd65c1f6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:59.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.135+0000 7fd6634f4640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd65c0d0230 0x7fd65c1e89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:59.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.135+0000 7fd6634f4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd65c1e8f00 0x7fd65c0cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:59.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.135+0000 7fd6634f4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd65c0d0d50 con 0x7fd65c1e8f00 2024-09-17T03:14:59.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.135+0000 7fd6634f4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd65c0b7d20 con 0x7fd65c0b7f10 2024-09-17T03:14:59.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.135+0000 7fd6634f4640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd65c0d0ed0 con 0x7fd65c0d0230 2024-09-17T03:14:59.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.135+0000 7fd661a91640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd65c1e8f00 0x7fd65c0cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:59.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.135+0000 7fd661a91640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd65c1e8f00 0x7fd65c0cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:39998/0 (socket says 172.21.15.87:39998) 2024-09-17T03:14:59.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.135+0000 7fd661a91640 1 -- 172.21.15.87:0/3665504417 learned_addr learned my addr 172.21.15.87:0/3665504417 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:59.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.135+0000 7fd661a91640 1 -- 172.21.15.87:0/3665504417 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd65c0d0230 msgr2=0x7fd65c1e89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:59.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.135+0000 7fd661a91640 1 --2- 172.21.15.87:0/3665504417 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd65c0d0230 0x7fd65c1e89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.135+0000 7fd661a91640 1 -- 172.21.15.87:0/3665504417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd65c0b7f10 msgr2=0x7fd65c1f6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:59.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.135+0000 7fd661290640 1 --2- 172.21.15.87:0/3665504417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd65c0b7f10 0x7fd65c1f6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:59.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.135+0000 7fd661a91640 1 --2- 172.21.15.87:0/3665504417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd65c0b7f10 0x7fd65c1f6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.135+0000 7fd661a91640 1 -- 172.21.15.87:0/3665504417 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd65c1553b0 con 0x7fd65c1e8f00 2024-09-17T03:14:59.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.136+0000 7fd661a91640 1 --2- 172.21.15.87:0/3665504417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd65c1e8f00 0x7fd65c0cb820 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd65400b440 tx=0x7fd65400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:59.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.136+0000 7fd651ffb640 1 -- 172.21.15.87:0/3665504417 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd654015020 con 0x7fd65c1e8f00 2024-09-17T03:14:59.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.136+0000 7fd651ffb640 1 -- 172.21.15.87:0/3665504417 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd654011920 con 0x7fd65c1e8f00 2024-09-17T03:14:59.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.136+0000 7fd6634f4640 1 -- 172.21.15.87:0/3665504417 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd65c1e7960 con 0x7fd65c1e8f00 2024-09-17T03:14:59.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.136+0000 7fd651ffb640 1 -- 172.21.15.87:0/3665504417 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd654011d70 con 0x7fd65c1e8f00 2024-09-17T03:14:59.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.136+0000 7fd6634f4640 1 -- 172.21.15.87:0/3665504417 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd65c1e7ee0 con 0x7fd65c1e8f00 2024-09-17T03:14:59.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.137+0000 7fd6634f4640 1 -- 172.21.15.87:0/3665504417 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd65c15d6e0 con 0x7fd65c1e8f00 2024-09-17T03:14:59.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.137+0000 7fd651ffb640 1 -- 172.21.15.87:0/3665504417 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd654018020 con 0x7fd65c1e8f00 2024-09-17T03:14:59.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.138+0000 7fd651ffb640 1 --2- 172.21.15.87:0/3665504417 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd640078bc0 0x7fd64007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:59.137 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.138+0000 7fd651ffb640 1 -- 172.21.15.87:0/3665504417 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fd65409a920 con 0x7fd65c1e8f00 2024-09-17T03:14:59.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.138+0000 7fd661290640 1 --2- 172.21.15.87:0/3665504417 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd640078bc0 0x7fd64007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:59.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.139+0000 7fd661290640 1 --2- 172.21.15.87:0/3665504417 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd640078bc0 0x7fd64007b0d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd658002020 tx=0x7fd658009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:59.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.142+0000 7fd651ffb640 1 -- 172.21.15.87:0/3665504417 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd65c15d6e0 con 0x7fd65c1e8f00 2024-09-17T03:14:59.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.311+0000 7fd6634f4640 1 -- 172.21.15.87:0/3665504417 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd65c15c5f0 con 0x7fd640078bc0 2024-09-17T03:14:59.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.316+0000 7fd651ffb640 1 -- 172.21.15.87:0/3665504417 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237587 (secure 0 0 0) 0x7fd65c15c5f0 con 0x7fd640078bc0 2024-09-17T03:14:59.317 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:59.318 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-09-17T03:14:59.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.319+0000 7fd6634f4640 1 -- 172.21.15.87:0/3665504417 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd640078bc0 msgr2=0x7fd64007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:59.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.319+0000 7fd6634f4640 1 --2- 172.21.15.87:0/3665504417 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd640078bc0 0x7fd64007b0d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd658002020 tx=0x7fd658009040 comp rx=0 tx=0).stop 2024-09-17T03:14:59.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.319+0000 7fd6634f4640 1 -- 172.21.15.87:0/3665504417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd65c1e8f00 msgr2=0x7fd65c0cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:59.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.319+0000 7fd6634f4640 1 --2- 172.21.15.87:0/3665504417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd65c1e8f00 0x7fd65c0cb820 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd65400b440 tx=0x7fd65400b900 comp rx=0 tx=0).stop 2024-09-17T03:14:59.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.319+0000 7fd6634f4640 1 -- 172.21.15.87:0/3665504417 shutdown_connections 2024-09-17T03:14:59.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.319+0000 7fd6634f4640 1 --2- 172.21.15.87:0/3665504417 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd640078bc0 0x7fd64007b0d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.319+0000 7fd6634f4640 1 --2- 172.21.15.87:0/3665504417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd65c1e8f00 0x7fd65c0cb820 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.319+0000 7fd6634f4640 1 --2- 172.21.15.87:0/3665504417 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd65c0d0230 0x7fd65c1e89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.319+0000 7fd6634f4640 1 --2- 172.21.15.87:0/3665504417 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd65c0b7f10 0x7fd65c1f6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.319+0000 7fd6634f4640 1 -- 172.21.15.87:0/3665504417 >> 172.21.15.87:0/3665504417 conn(0x7fd65c069330 msgr2=0x7fd65c15fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:59.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.319+0000 7fd6634f4640 1 -- 172.21.15.87:0/3665504417 shutdown_connections 2024-09-17T03:14:59.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.319+0000 7fd6634f4640 1 -- 172.21.15.87:0/3665504417 wait complete. 2024-09-17T03:14:59.332 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2024-09-17T03:14:57.426719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":12,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":68,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":224188,"kb_used_data":3720,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750532,"statfs":{"total":773094113280,"available":772864544768,"internally_reserved":0,"allocated":3809280,"data_stored":2141856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000893"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T03:14:54.312526+0000","last_change":"2024-09-17T03:14:54.312526+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.312526+0000","last_undegraded":"2024-09-17T03:14:54.312526+0000","last_fullsized":"2024-09-17T03:14:54.312526+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T03:14:54.312074+0000","last_change":"2024-09-17T03:14:54.312074+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.312074+0000","last_undegraded":"2024-09-17T03:14:54.312074+0000","last_fullsized":"2024-09-17T03:14:54.312074+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T03:14:54.311860+0000","last_change":"2024-09-17T03:14:54.311860+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.311860+0000","last_undegraded":"2024-09-17T03:14:54.311860+0000","last_fullsized":"2024-09-17T03:14:54.311860+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T03:14:54.313012+0000","last_change":"2024-09-17T03:14:54.313012+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.313012+0000","last_undegraded":"2024-09-17T03:14:54.313012+0000","last_fullsized":"2024-09-17T03:14:54.313012+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T03:14:54.312546+0000","last_change":"2024-09-17T03:14:54.312546+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.312546+0000","last_undegraded":"2024-09-17T03:14:54.312546+0000","last_fullsized":"2024-09-17T03:14:54.312546+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:54.302142+0000","last_change":"2024-09-17T03:14:54.302142+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.302142+0000","last_undegraded":"2024-09-17T03:14:54.302142+0000","last_fullsized":"2024-09-17T03:14:54.302142+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":121,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-17T03:14:51.082180+0000","last_change":"2024-09-17T03:14:04.862167+0000","last_active":"2024-09-17T03:14:51.082180+0000","last_peered":"2024-09-17T03:14:51.082180+0000","last_clean":"2024-09-17T03:14:51.082180+0000","last_became_active":"2024-09-17T03:14:04.861731+0000","last_became_peered":"2024-09-17T03:14:04.861731+0000","last_unstale":"2024-09-17T03:14:51.082180+0000","last_undegraded":"2024-09-17T03:14:51.082180+0000","last_fullsized":"2024-09-17T03:14:51.082180+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_clean_scrub_stamp":"2024-09-17T03:14:03.847721+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:32.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":10,"acting":10,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509455,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27364,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":327680,"data_stored":120140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27356,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":327680,"data_stored":120140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509454,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32104,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339736,"statfs":{"total":96636764160,"available":96603889664,"internally_reserved":0,"allocated":921600,"data_stored":710508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27352,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":327680,"data_stored":120140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27356,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":327680,"data_stored":120140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":921600,"data_stored":710508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509454,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27360,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":327680,"data_stored":120140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":327680,"data_stored":120140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:14:59.337 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:14:59.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.484+0000 7f35ebc69640 1 -- 172.21.15.87:0/1372094034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e415f0c0 msgr2=0x7f35e40748a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:59.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.484+0000 7f35ebc69640 1 --2- 172.21.15.87:0/1372094034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e415f0c0 0x7f35e40748a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f35e4072ca0 tx=0x7f35d402a560 comp rx=0 tx=0).stop 2024-09-17T03:14:59.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.484+0000 7f35ebc69640 1 -- 172.21.15.87:0/1372094034 shutdown_connections 2024-09-17T03:14:59.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.484+0000 7f35ebc69640 1 --2- 172.21.15.87:0/1372094034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e415f0c0 0x7f35e40748a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.484+0000 7f35ebc69640 1 --2- 172.21.15.87:0/1372094034 >> v2:172.21.15.149:3300/0 conn(0x7f35e415f4d0 0x7f35e4074de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.484+0000 7f35ebc69640 1 --2- 172.21.15.87:0/1372094034 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35e40cc590 0x7f35e415c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.484+0000 7f35ebc69640 1 -- 172.21.15.87:0/1372094034 >> 172.21.15.87:0/1372094034 conn(0x7f35e406ee70 msgr2=0x7f35e406f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:59.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.484+0000 7f35ebc69640 1 -- 172.21.15.87:0/1372094034 shutdown_connections 2024-09-17T03:14:59.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.484+0000 7f35ebc69640 1 -- 172.21.15.87:0/1372094034 wait complete. 2024-09-17T03:14:59.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.486+0000 7f35ebc69640 1 Processor -- start 2024-09-17T03:14:59.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.486+0000 7f35ebc69640 1 -- start start 2024-09-17T03:14:59.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.487+0000 7f35ebc69640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e40cc590 0x7f35e414e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:59.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.487+0000 7f35ebc69640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35e415f0c0 0x7f35e4152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:59.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.487+0000 7f35ebc69640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35e415f4d0 0x7f35e40b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:59.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.487+0000 7f35ebc69640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f35e40c4f00 con 0x7f35e40cc590 2024-09-17T03:14:59.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.487+0000 7f35ebc69640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f35e40c6b50 con 0x7f35e415f0c0 2024-09-17T03:14:59.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.487+0000 7f35ebc69640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f35e40c5080 con 0x7f35e415f4d0 2024-09-17T03:14:59.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.487+0000 7f35e9a05640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e40cc590 0x7f35e414e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:59.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.487+0000 7f35e9a05640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e40cc590 0x7f35e414e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40008/0 (socket says 172.21.15.87:40008) 2024-09-17T03:14:59.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.487+0000 7f35e9a05640 1 -- 172.21.15.87:0/2284876154 learned_addr learned my addr 172.21.15.87:0/2284876154 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:59.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.487+0000 7f35ea206640 1 --2- 172.21.15.87:0/2284876154 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35e415f4d0 0x7f35e40b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:59.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.487+0000 7f35e9204640 1 --2- 172.21.15.87:0/2284876154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35e415f0c0 0x7f35e4152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:59.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.488+0000 7f35e9a05640 1 -- 172.21.15.87:0/2284876154 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35e415f4d0 msgr2=0x7f35e40b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:59.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.488+0000 7f35e9a05640 1 --2- 172.21.15.87:0/2284876154 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35e415f4d0 0x7f35e40b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.488+0000 7f35e9a05640 1 -- 172.21.15.87:0/2284876154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35e415f0c0 msgr2=0x7f35e4152270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:59.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.488+0000 7f35e9a05640 1 --2- 172.21.15.87:0/2284876154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35e415f0c0 0x7f35e4152270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.488+0000 7f35e9a05640 1 -- 172.21.15.87:0/2284876154 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e41f3c50 con 0x7f35e40cc590 2024-09-17T03:14:59.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.488+0000 7f35e9a05640 1 --2- 172.21.15.87:0/2284876154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e40cc590 0x7f35e414e9e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f35e000d750 tx=0x7f35e000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:59.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.488+0000 7f35da7fc640 1 -- 172.21.15.87:0/2284876154 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0017070 con 0x7f35e40cc590 2024-09-17T03:14:59.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.488+0000 7f35da7fc640 1 -- 172.21.15.87:0/2284876154 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35e0009b20 con 0x7f35e40cc590 2024-09-17T03:14:59.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.488+0000 7f35ebc69640 1 -- 172.21.15.87:0/2284876154 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e41f3f30 con 0x7f35e40cc590 2024-09-17T03:14:59.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.488+0000 7f35da7fc640 1 -- 172.21.15.87:0/2284876154 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e0010460 con 0x7f35e40cc590 2024-09-17T03:14:59.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.488+0000 7f35ebc69640 1 -- 172.21.15.87:0/2284876154 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e41f44f0 con 0x7f35e40cc590 2024-09-17T03:14:59.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.489+0000 7f35ebc69640 1 -- 172.21.15.87:0/2284876154 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e415c470 con 0x7f35e40cc590 2024-09-17T03:14:59.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.490+0000 7f35da7fc640 1 -- 172.21.15.87:0/2284876154 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f35e001a070 con 0x7f35e40cc590 2024-09-17T03:14:59.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.491+0000 7f35da7fc640 1 --2- 172.21.15.87:0/2284876154 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f35c8078b90 0x7f35c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:59.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.491+0000 7f35da7fc640 1 -- 172.21.15.87:0/2284876154 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f35e009af00 con 0x7f35e40cc590 2024-09-17T03:14:59.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.491+0000 7f35e9204640 1 --2- 172.21.15.87:0/2284876154 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f35c8078b90 0x7f35c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:59.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.491+0000 7f35e9204640 1 --2- 172.21.15.87:0/2284876154 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f35c8078b90 0x7f35c807b0a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f35d4033000 tx=0x7f35d4004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:59.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.495+0000 7f35da7fc640 1 -- 172.21.15.87:0/2284876154 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f35e0063340 con 0x7f35e40cc590 2024-09-17T03:14:59.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.664+0000 7f35ebc69640 1 -- 172.21.15.87:0/2284876154 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f35e414f0c0 con 0x7f35c8078b90 2024-09-17T03:14:59.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.669+0000 7f35da7fc640 1 -- 172.21.15.87:0/2284876154 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249913 (secure 0 0 0) 0x7f35e414f0c0 con 0x7f35c8078b90 2024-09-17T03:14:59.670 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:14:59.671 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-09-17T03:14:59.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.672+0000 7f35ebc69640 1 -- 172.21.15.87:0/2284876154 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f35c8078b90 msgr2=0x7f35c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:59.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.672+0000 7f35ebc69640 1 --2- 172.21.15.87:0/2284876154 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f35c8078b90 0x7f35c807b0a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f35d4033000 tx=0x7f35d4004730 comp rx=0 tx=0).stop 2024-09-17T03:14:59.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.672+0000 7f35ebc69640 1 -- 172.21.15.87:0/2284876154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e40cc590 msgr2=0x7f35e414e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:59.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.672+0000 7f35ebc69640 1 --2- 172.21.15.87:0/2284876154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e40cc590 0x7f35e414e9e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f35e000d750 tx=0x7f35e000dc10 comp rx=0 tx=0).stop 2024-09-17T03:14:59.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.672+0000 7f35ebc69640 1 -- 172.21.15.87:0/2284876154 shutdown_connections 2024-09-17T03:14:59.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.672+0000 7f35ebc69640 1 --2- 172.21.15.87:0/2284876154 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f35c8078b90 0x7f35c807b0a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.672+0000 7f35ebc69640 1 --2- 172.21.15.87:0/2284876154 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35e415f4d0 0x7f35e40b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.672+0000 7f35ebc69640 1 --2- 172.21.15.87:0/2284876154 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35e415f0c0 0x7f35e4152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.672+0000 7f35ebc69640 1 --2- 172.21.15.87:0/2284876154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35e40cc590 0x7f35e414e9e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.672+0000 7f35ebc69640 1 -- 172.21.15.87:0/2284876154 >> 172.21.15.87:0/2284876154 conn(0x7f35e406ee70 msgr2=0x7f35e415cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:59.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.672+0000 7f35ebc69640 1 -- 172.21.15.87:0/2284876154 shutdown_connections 2024-09-17T03:14:59.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.672+0000 7f35ebc69640 1 -- 172.21.15.87:0/2284876154 wait complete. 2024-09-17T03:14:59.682 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-09-17T03:14:59.427221+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":142,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":170,"acting":170,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":94,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224464,"kb_used_data":4020,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750256,"statfs":{"total":773094113280,"available":772864262144,"internally_reserved":0,"allocated":4116480,"data_stored":2277111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.117237"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351505+0000","last_change":"2024-09-17T03:14:56.344756+0000","last_active":"2024-09-17T03:14:56.351505+0000","last_peered":"2024-09-17T03:14:56.351505+0000","last_clean":"2024-09-17T03:14:56.351505+0000","last_became_active":"2024-09-17T03:14:56.343497+0000","last_became_peered":"2024-09-17T03:14:56.343497+0000","last_unstale":"2024-09-17T03:14:56.351505+0000","last_undegraded":"2024-09-17T03:14:56.351505+0000","last_fullsized":"2024-09-17T03:14:56.351505+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:07.377313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344120+0000","last_change":"2024-09-17T03:14:56.348503+0000","last_active":"2024-09-17T03:14:57.344120+0000","last_peered":"2024-09-17T03:14:57.344120+0000","last_clean":"2024-09-17T03:14:57.344120+0000","last_became_active":"2024-09-17T03:14:56.348172+0000","last_became_peered":"2024-09-17T03:14:56.348172+0000","last_unstale":"2024-09-17T03:14:57.344120+0000","last_undegraded":"2024-09-17T03:14:57.344120+0000","last_fullsized":"2024-09-17T03:14:57.344120+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:52:09.125634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.349842+0000","last_change":"2024-09-17T03:14:56.347013+0000","last_active":"2024-09-17T03:14:56.349842+0000","last_peered":"2024-09-17T03:14:56.349842+0000","last_clean":"2024-09-17T03:14:56.349842+0000","last_became_active":"2024-09-17T03:14:56.346718+0000","last_became_peered":"2024-09-17T03:14:56.346718+0000","last_unstale":"2024-09-17T03:14:56.349842+0000","last_undegraded":"2024-09-17T03:14:56.349842+0000","last_fullsized":"2024-09-17T03:14:56.349842+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:17.382050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.343763+0000","last_change":"2024-09-17T03:14:56.344528+0000","last_active":"2024-09-17T03:14:57.343763+0000","last_peered":"2024-09-17T03:14:57.343763+0000","last_clean":"2024-09-17T03:14:57.343763+0000","last_became_active":"2024-09-17T03:14:56.344226+0000","last_became_peered":"2024-09-17T03:14:56.344226+0000","last_unstale":"2024-09-17T03:14:57.343763+0000","last_undegraded":"2024-09-17T03:14:57.343763+0000","last_fullsized":"2024-09-17T03:14:57.343763+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:09:48.039023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.340976+0000","last_change":"2024-09-17T03:14:56.342382+0000","last_active":"2024-09-17T03:14:57.340976+0000","last_peered":"2024-09-17T03:14:57.340976+0000","last_clean":"2024-09-17T03:14:57.340976+0000","last_became_active":"2024-09-17T03:14:56.341980+0000","last_became_peered":"2024-09-17T03:14:56.341980+0000","last_unstale":"2024-09-17T03:14:57.340976+0000","last_undegraded":"2024-09-17T03:14:57.340976+0000","last_fullsized":"2024-09-17T03:14:57.340976+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:11.714679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.345446+0000","last_change":"2024-09-17T03:14:56.343019+0000","last_active":"2024-09-17T03:14:56.345446+0000","last_peered":"2024-09-17T03:14:56.345446+0000","last_clean":"2024-09-17T03:14:56.345446+0000","last_became_active":"2024-09-17T03:14:56.342705+0000","last_became_peered":"2024-09-17T03:14:56.342705+0000","last_unstale":"2024-09-17T03:14:56.345446+0000","last_undegraded":"2024-09-17T03:14:56.345446+0000","last_fullsized":"2024-09-17T03:14:56.345446+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:04.002930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.352770+0000","last_change":"2024-09-17T03:14:56.346692+0000","last_active":"2024-09-17T03:14:56.352770+0000","last_peered":"2024-09-17T03:14:56.352770+0000","last_clean":"2024-09-17T03:14:56.352770+0000","last_became_active":"2024-09-17T03:14:56.346397+0000","last_became_peered":"2024-09-17T03:14:56.346397+0000","last_unstale":"2024-09-17T03:14:56.352770+0000","last_undegraded":"2024-09-17T03:14:56.352770+0000","last_fullsized":"2024-09-17T03:14:56.352770+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:28.743694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.344785+0000","last_change":"2024-09-17T03:14:56.339990+0000","last_active":"2024-09-17T03:14:56.344785+0000","last_peered":"2024-09-17T03:14:56.344785+0000","last_clean":"2024-09-17T03:14:56.344785+0000","last_became_active":"2024-09-17T03:14:56.339281+0000","last_became_peered":"2024-09-17T03:14:56.339281+0000","last_unstale":"2024-09-17T03:14:56.344785+0000","last_undegraded":"2024-09-17T03:14:56.344785+0000","last_fullsized":"2024-09-17T03:14:56.344785+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:32:19.770373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351104+0000","last_change":"2024-09-17T03:14:56.342266+0000","last_active":"2024-09-17T03:14:56.351104+0000","last_peered":"2024-09-17T03:14:56.351104+0000","last_clean":"2024-09-17T03:14:56.351104+0000","last_became_active":"2024-09-17T03:14:56.341978+0000","last_became_peered":"2024-09-17T03:14:56.341978+0000","last_unstale":"2024-09-17T03:14:56.351104+0000","last_undegraded":"2024-09-17T03:14:56.351104+0000","last_fullsized":"2024-09-17T03:14:56.351104+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:50:28.343075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.345419+0000","last_change":"2024-09-17T03:14:56.342630+0000","last_active":"2024-09-17T03:14:56.345419+0000","last_peered":"2024-09-17T03:14:56.345419+0000","last_clean":"2024-09-17T03:14:56.345419+0000","last_became_active":"2024-09-17T03:14:56.342175+0000","last_became_peered":"2024-09-17T03:14:56.342175+0000","last_unstale":"2024-09-17T03:14:56.345419+0000","last_undegraded":"2024-09-17T03:14:56.345419+0000","last_fullsized":"2024-09-17T03:14:56.345419+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:02:14.647806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.357637+0000","last_change":"2024-09-17T03:14:56.351892+0000","last_active":"2024-09-17T03:14:56.357637+0000","last_peered":"2024-09-17T03:14:56.357637+0000","last_clean":"2024-09-17T03:14:56.357637+0000","last_became_active":"2024-09-17T03:14:56.351149+0000","last_became_peered":"2024-09-17T03:14:56.351149+0000","last_unstale":"2024-09-17T03:14:56.357637+0000","last_undegraded":"2024-09-17T03:14:56.357637+0000","last_fullsized":"2024-09-17T03:14:56.357637+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:14:16.498828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351551+0000","last_change":"2024-09-17T03:14:56.349422+0000","last_active":"2024-09-17T03:14:56.351551+0000","last_peered":"2024-09-17T03:14:56.351551+0000","last_clean":"2024-09-17T03:14:56.351551+0000","last_became_active":"2024-09-17T03:14:56.349132+0000","last_became_peered":"2024-09-17T03:14:56.349132+0000","last_unstale":"2024-09-17T03:14:56.351551+0000","last_undegraded":"2024-09-17T03:14:56.351551+0000","last_fullsized":"2024-09-17T03:14:56.351551+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:27.164191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.340907+0000","last_change":"2024-09-17T03:14:56.337002+0000","last_active":"2024-09-17T03:14:56.340907+0000","last_peered":"2024-09-17T03:14:56.340907+0000","last_clean":"2024-09-17T03:14:56.340907+0000","last_became_active":"2024-09-17T03:14:56.336649+0000","last_became_peered":"2024-09-17T03:14:56.336649+0000","last_unstale":"2024-09-17T03:14:56.340907+0000","last_undegraded":"2024-09-17T03:14:56.340907+0000","last_fullsized":"2024-09-17T03:14:56.340907+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:02:48.048417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.354704+0000","last_change":"2024-09-17T03:14:56.346405+0000","last_active":"2024-09-17T03:14:56.354704+0000","last_peered":"2024-09-17T03:14:56.354704+0000","last_clean":"2024-09-17T03:14:56.354704+0000","last_became_active":"2024-09-17T03:14:56.346037+0000","last_became_peered":"2024-09-17T03:14:56.346037+0000","last_unstale":"2024-09-17T03:14:56.354704+0000","last_undegraded":"2024-09-17T03:14:56.354704+0000","last_fullsized":"2024-09-17T03:14:56.354704+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:04:18.618725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341208+0000","last_change":"2024-09-17T03:14:56.345147+0000","last_active":"2024-09-17T03:14:57.341208+0000","last_peered":"2024-09-17T03:14:57.341208+0000","last_clean":"2024-09-17T03:14:57.341208+0000","last_became_active":"2024-09-17T03:14:56.344844+0000","last_became_peered":"2024-09-17T03:14:56.344844+0000","last_unstale":"2024-09-17T03:14:57.341208+0000","last_undegraded":"2024-09-17T03:14:57.341208+0000","last_fullsized":"2024-09-17T03:14:57.341208+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:30:40.186081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341134+0000","last_change":"2024-09-17T03:14:56.339650+0000","last_active":"2024-09-17T03:14:57.341134+0000","last_peered":"2024-09-17T03:14:57.341134+0000","last_clean":"2024-09-17T03:14:57.341134+0000","last_became_active":"2024-09-17T03:14:56.338719+0000","last_became_peered":"2024-09-17T03:14:56.338719+0000","last_unstale":"2024-09-17T03:14:57.341134+0000","last_undegraded":"2024-09-17T03:14:57.341134+0000","last_fullsized":"2024-09-17T03:14:57.341134+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:51:33.167537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351809+0000","last_change":"2024-09-17T03:14:56.341270+0000","last_active":"2024-09-17T03:14:56.351809+0000","last_peered":"2024-09-17T03:14:56.351809+0000","last_clean":"2024-09-17T03:14:56.351809+0000","last_became_active":"2024-09-17T03:14:56.340876+0000","last_became_peered":"2024-09-17T03:14:56.340876+0000","last_unstale":"2024-09-17T03:14:56.351809+0000","last_undegraded":"2024-09-17T03:14:56.351809+0000","last_fullsized":"2024-09-17T03:14:56.351809+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:07.377313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.349912+0000","last_change":"2024-09-17T03:14:56.344410+0000","last_active":"2024-09-17T03:14:56.349912+0000","last_peered":"2024-09-17T03:14:56.349912+0000","last_clean":"2024-09-17T03:14:56.349912+0000","last_became_active":"2024-09-17T03:14:56.344151+0000","last_became_peered":"2024-09-17T03:14:56.344151+0000","last_unstale":"2024-09-17T03:14:56.349912+0000","last_undegraded":"2024-09-17T03:14:56.349912+0000","last_fullsized":"2024-09-17T03:14:56.349912+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:38:12.850097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.356153+0000","last_change":"2024-09-17T03:14:56.354343+0000","last_active":"2024-09-17T03:14:56.356153+0000","last_peered":"2024-09-17T03:14:56.356153+0000","last_clean":"2024-09-17T03:14:56.356153+0000","last_became_active":"2024-09-17T03:14:56.354020+0000","last_became_peered":"2024-09-17T03:14:56.354020+0000","last_unstale":"2024-09-17T03:14:56.356153+0000","last_undegraded":"2024-09-17T03:14:56.356153+0000","last_fullsized":"2024-09-17T03:14:56.356153+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:23.062711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351758+0000","last_change":"2024-09-17T03:14:56.340466+0000","last_active":"2024-09-17T03:14:56.351758+0000","last_peered":"2024-09-17T03:14:56.351758+0000","last_clean":"2024-09-17T03:14:56.351758+0000","last_became_active":"2024-09-17T03:14:56.340135+0000","last_became_peered":"2024-09-17T03:14:56.340135+0000","last_unstale":"2024-09-17T03:14:56.351758+0000","last_undegraded":"2024-09-17T03:14:56.351758+0000","last_fullsized":"2024-09-17T03:14:56.351758+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:05.079987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.356113+0000","last_change":"2024-09-17T03:14:56.353500+0000","last_active":"2024-09-17T03:14:56.356113+0000","last_peered":"2024-09-17T03:14:56.356113+0000","last_clean":"2024-09-17T03:14:56.356113+0000","last_became_active":"2024-09-17T03:14:56.353211+0000","last_became_peered":"2024-09-17T03:14:56.353211+0000","last_unstale":"2024-09-17T03:14:56.356113+0000","last_undegraded":"2024-09-17T03:14:56.356113+0000","last_fullsized":"2024-09-17T03:14:56.356113+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:11.473585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.343839+0000","last_change":"2024-09-17T03:14:56.329704+0000","last_active":"2024-09-17T03:14:57.343839+0000","last_peered":"2024-09-17T03:14:57.343839+0000","last_clean":"2024-09-17T03:14:57.343839+0000","last_became_active":"2024-09-17T03:14:56.329354+0000","last_became_peered":"2024-09-17T03:14:56.329354+0000","last_unstale":"2024-09-17T03:14:57.343839+0000","last_undegraded":"2024-09-17T03:14:57.343839+0000","last_fullsized":"2024-09-17T03:14:57.343839+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:59:50.923956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.352937+0000","last_change":"2024-09-17T03:14:56.348309+0000","last_active":"2024-09-17T03:14:56.352937+0000","last_peered":"2024-09-17T03:14:56.352937+0000","last_clean":"2024-09-17T03:14:56.352937+0000","last_became_active":"2024-09-17T03:14:56.347990+0000","last_became_peered":"2024-09-17T03:14:56.347990+0000","last_unstale":"2024-09-17T03:14:56.352937+0000","last_undegraded":"2024-09-17T03:14:56.352937+0000","last_fullsized":"2024-09-17T03:14:56.352937+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:25.974302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.348883+0000","last_change":"2024-09-17T03:14:56.342697+0000","last_active":"2024-09-17T03:14:56.348883+0000","last_peered":"2024-09-17T03:14:56.348883+0000","last_clean":"2024-09-17T03:14:56.348883+0000","last_became_active":"2024-09-17T03:14:56.342375+0000","last_became_peered":"2024-09-17T03:14:56.342375+0000","last_unstale":"2024-09-17T03:14:56.348883+0000","last_undegraded":"2024-09-17T03:14:56.348883+0000","last_fullsized":"2024-09-17T03:14:56.348883+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:04.426421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.350990+0000","last_change":"2024-09-17T03:14:56.346711+0000","last_active":"2024-09-17T03:14:56.350990+0000","last_peered":"2024-09-17T03:14:56.350990+0000","last_clean":"2024-09-17T03:14:56.350990+0000","last_became_active":"2024-09-17T03:14:56.346399+0000","last_became_peered":"2024-09-17T03:14:56.346399+0000","last_unstale":"2024-09-17T03:14:56.350990+0000","last_undegraded":"2024-09-17T03:14:56.350990+0000","last_fullsized":"2024-09-17T03:14:56.350990+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:20.900685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.352810+0000","last_change":"2024-09-17T03:14:56.348432+0000","last_active":"2024-09-17T03:14:56.352810+0000","last_peered":"2024-09-17T03:14:56.352810+0000","last_clean":"2024-09-17T03:14:56.352810+0000","last_became_active":"2024-09-17T03:14:56.348048+0000","last_became_peered":"2024-09-17T03:14:56.348048+0000","last_unstale":"2024-09-17T03:14:56.352810+0000","last_undegraded":"2024-09-17T03:14:56.352810+0000","last_fullsized":"2024-09-17T03:14:56.352810+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:26:13.257738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351063+0000","last_change":"2024-09-17T03:14:56.346523+0000","last_active":"2024-09-17T03:14:56.351063+0000","last_peered":"2024-09-17T03:14:56.351063+0000","last_clean":"2024-09-17T03:14:56.351063+0000","last_became_active":"2024-09-17T03:14:56.346207+0000","last_became_peered":"2024-09-17T03:14:56.346207+0000","last_unstale":"2024-09-17T03:14:56.351063+0000","last_undegraded":"2024-09-17T03:14:56.351063+0000","last_fullsized":"2024-09-17T03:14:56.351063+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:53.559531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351860+0000","last_change":"2024-09-17T03:14:56.341373+0000","last_active":"2024-09-17T03:14:56.351860+0000","last_peered":"2024-09-17T03:14:56.351860+0000","last_clean":"2024-09-17T03:14:56.351860+0000","last_became_active":"2024-09-17T03:14:56.341066+0000","last_became_peered":"2024-09-17T03:14:56.341066+0000","last_unstale":"2024-09-17T03:14:56.351860+0000","last_undegraded":"2024-09-17T03:14:56.351860+0000","last_fullsized":"2024-09-17T03:14:56.351860+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:28.743694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.354608+0000","last_change":"2024-09-17T03:14:55.321527+0000","last_active":"2024-09-17T03:14:56.354608+0000","last_peered":"2024-09-17T03:14:56.354608+0000","last_clean":"2024-09-17T03:14:56.354608+0000","last_became_active":"2024-09-17T03:14:55.320714+0000","last_became_peered":"2024-09-17T03:14:55.320714+0000","last_unstale":"2024-09-17T03:14:56.354608+0000","last_undegraded":"2024-09-17T03:14:56.354608+0000","last_fullsized":"2024-09-17T03:14:56.354608+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:16:27.821016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351470+0000","last_change":"2024-09-17T03:14:56.344463+0000","last_active":"2024-09-17T03:14:56.351470+0000","last_peered":"2024-09-17T03:14:56.351470+0000","last_clean":"2024-09-17T03:14:56.351470+0000","last_became_active":"2024-09-17T03:14:56.344125+0000","last_became_peered":"2024-09-17T03:14:56.344125+0000","last_unstale":"2024-09-17T03:14:56.351470+0000","last_undegraded":"2024-09-17T03:14:56.351470+0000","last_fullsized":"2024-09-17T03:14:56.351470+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:05.079987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T03:14:54.312526+0000","last_change":"2024-09-17T03:14:54.312526+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.312526+0000","last_undegraded":"2024-09-17T03:14:54.312526+0000","last_fullsized":"2024-09-17T03:14:54.312526+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.356182+0000","last_change":"2024-09-17T03:14:56.354666+0000","last_active":"2024-09-17T03:14:56.356182+0000","last_peered":"2024-09-17T03:14:56.356182+0000","last_clean":"2024-09-17T03:14:56.356182+0000","last_became_active":"2024-09-17T03:14:56.354360+0000","last_became_peered":"2024-09-17T03:14:56.354360+0000","last_unstale":"2024-09-17T03:14:56.356182+0000","last_undegraded":"2024-09-17T03:14:56.356182+0000","last_fullsized":"2024-09-17T03:14:56.356182+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:19.990834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341309+0000","last_change":"2024-09-17T03:14:55.332084+0000","last_active":"2024-09-17T03:14:57.341309+0000","last_peered":"2024-09-17T03:14:57.341309+0000","last_clean":"2024-09-17T03:14:57.341309+0000","last_became_active":"2024-09-17T03:14:55.331735+0000","last_became_peered":"2024-09-17T03:14:55.331735+0000","last_unstale":"2024-09-17T03:14:57.341309+0000","last_undegraded":"2024-09-17T03:14:57.341309+0000","last_fullsized":"2024-09-17T03:14:57.341309+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:23.002081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T03:14:54.312074+0000","last_change":"2024-09-17T03:14:54.312074+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.312074+0000","last_undegraded":"2024-09-17T03:14:54.312074+0000","last_fullsized":"2024-09-17T03:14:54.312074+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.348195+0000","last_change":"2024-09-17T03:14:56.345924+0000","last_active":"2024-09-17T03:14:56.348195+0000","last_peered":"2024-09-17T03:14:56.348195+0000","last_clean":"2024-09-17T03:14:56.348195+0000","last_became_active":"2024-09-17T03:14:56.345340+0000","last_became_peered":"2024-09-17T03:14:56.345340+0000","last_unstale":"2024-09-17T03:14:56.348195+0000","last_undegraded":"2024-09-17T03:14:56.348195+0000","last_fullsized":"2024-09-17T03:14:56.348195+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:18.485654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.357022+0000","last_change":"2024-09-17T03:14:55.334211+0000","last_active":"2024-09-17T03:14:56.357022+0000","last_peered":"2024-09-17T03:14:56.357022+0000","last_clean":"2024-09-17T03:14:56.357022+0000","last_became_active":"2024-09-17T03:14:55.333714+0000","last_became_peered":"2024-09-17T03:14:55.333714+0000","last_unstale":"2024-09-17T03:14:56.357022+0000","last_undegraded":"2024-09-17T03:14:56.357022+0000","last_fullsized":"2024-09-17T03:14:56.357022+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:30:39.180289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.353185+0000","last_change":"2024-09-17T03:14:56.348919+0000","last_active":"2024-09-17T03:14:56.353185+0000","last_peered":"2024-09-17T03:14:56.353185+0000","last_clean":"2024-09-17T03:14:56.353185+0000","last_became_active":"2024-09-17T03:14:56.348235+0000","last_became_peered":"2024-09-17T03:14:56.348235+0000","last_unstale":"2024-09-17T03:14:56.353185+0000","last_undegraded":"2024-09-17T03:14:56.353185+0000","last_fullsized":"2024-09-17T03:14:56.353185+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:14:16.498828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.357606+0000","last_change":"2024-09-17T03:14:55.335181+0000","last_active":"2024-09-17T03:14:56.357606+0000","last_peered":"2024-09-17T03:14:56.357606+0000","last_clean":"2024-09-17T03:14:56.357606+0000","last_became_active":"2024-09-17T03:14:55.334860+0000","last_became_peered":"2024-09-17T03:14:55.334860+0000","last_unstale":"2024-09-17T03:14:56.357606+0000","last_undegraded":"2024-09-17T03:14:56.357606+0000","last_fullsized":"2024-09-17T03:14:56.357606+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:34:41.387404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.341334+0000","last_change":"2024-09-17T03:14:56.338147+0000","last_active":"2024-09-17T03:14:56.341334+0000","last_peered":"2024-09-17T03:14:56.341334+0000","last_clean":"2024-09-17T03:14:56.341334+0000","last_became_active":"2024-09-17T03:14:56.337801+0000","last_became_peered":"2024-09-17T03:14:56.337801+0000","last_unstale":"2024-09-17T03:14:56.341334+0000","last_undegraded":"2024-09-17T03:14:56.341334+0000","last_fullsized":"2024-09-17T03:14:56.341334+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:24.007873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.340815+0000","last_change":"2024-09-17T03:14:55.328132+0000","last_active":"2024-09-17T03:14:57.340815+0000","last_peered":"2024-09-17T03:14:57.340815+0000","last_clean":"2024-09-17T03:14:57.340815+0000","last_became_active":"2024-09-17T03:14:55.327773+0000","last_became_peered":"2024-09-17T03:14:55.327773+0000","last_unstale":"2024-09-17T03:14:57.340815+0000","last_undegraded":"2024-09-17T03:14:57.340815+0000","last_fullsized":"2024-09-17T03:14:57.340815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:09.899391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.330123+0000","last_change":"2024-09-17T03:14:55.334322+0000","last_active":"2024-09-17T03:14:56.330123+0000","last_peered":"2024-09-17T03:14:56.330123+0000","last_clean":"2024-09-17T03:14:56.330123+0000","last_became_active":"2024-09-17T03:14:55.333947+0000","last_became_peered":"2024-09-17T03:14:55.333947+0000","last_unstale":"2024-09-17T03:14:56.330123+0000","last_undegraded":"2024-09-17T03:14:56.330123+0000","last_fullsized":"2024-09-17T03:14:56.330123+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:44.089548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351133+0000","last_change":"2024-09-17T03:14:56.345419+0000","last_active":"2024-09-17T03:14:56.351133+0000","last_peered":"2024-09-17T03:14:56.351133+0000","last_clean":"2024-09-17T03:14:56.351133+0000","last_became_active":"2024-09-17T03:14:56.343640+0000","last_became_peered":"2024-09-17T03:14:56.343640+0000","last_unstale":"2024-09-17T03:14:56.351133+0000","last_undegraded":"2024-09-17T03:14:56.351133+0000","last_fullsized":"2024-09-17T03:14:56.351133+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:20.330393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344278+0000","last_change":"2024-09-17T03:14:55.323288+0000","last_active":"2024-09-17T03:14:57.344278+0000","last_peered":"2024-09-17T03:14:57.344278+0000","last_clean":"2024-09-17T03:14:57.344278+0000","last_became_active":"2024-09-17T03:14:55.322723+0000","last_became_peered":"2024-09-17T03:14:55.322723+0000","last_unstale":"2024-09-17T03:14:57.344278+0000","last_undegraded":"2024-09-17T03:14:57.344278+0000","last_fullsized":"2024-09-17T03:14:57.344278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:11:24.012880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344030+0000","last_change":"2024-09-17T03:14:55.325887+0000","last_active":"2024-09-17T03:14:57.344030+0000","last_peered":"2024-09-17T03:14:57.344030+0000","last_clean":"2024-09-17T03:14:57.344030+0000","last_became_active":"2024-09-17T03:14:55.325563+0000","last_became_peered":"2024-09-17T03:14:55.325563+0000","last_unstale":"2024-09-17T03:14:57.344030+0000","last_undegraded":"2024-09-17T03:14:57.344030+0000","last_fullsized":"2024-09-17T03:14:57.344030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:22.517631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.354205+0000","last_change":"2024-09-17T03:14:55.329954+0000","last_active":"2024-09-17T03:14:56.354205+0000","last_peered":"2024-09-17T03:14:56.354205+0000","last_clean":"2024-09-17T03:14:56.354205+0000","last_became_active":"2024-09-17T03:14:55.329631+0000","last_became_peered":"2024-09-17T03:14:55.329631+0000","last_unstale":"2024-09-17T03:14:56.354205+0000","last_undegraded":"2024-09-17T03:14:56.354205+0000","last_fullsized":"2024-09-17T03:14:56.354205+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:45.028465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.354624+0000","last_change":"2024-09-17T03:14:56.352353+0000","last_active":"2024-09-17T03:14:56.354624+0000","last_peered":"2024-09-17T03:14:56.354624+0000","last_clean":"2024-09-17T03:14:56.354624+0000","last_became_active":"2024-09-17T03:14:56.352041+0000","last_became_peered":"2024-09-17T03:14:56.352041+0000","last_unstale":"2024-09-17T03:14:56.354624+0000","last_undegraded":"2024-09-17T03:14:56.354624+0000","last_fullsized":"2024-09-17T03:14:56.354624+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:43:07.926923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.360502+0000","last_change":"2024-09-17T03:14:55.332350+0000","last_active":"2024-09-17T03:14:56.360502+0000","last_peered":"2024-09-17T03:14:56.360502+0000","last_clean":"2024-09-17T03:14:56.360502+0000","last_became_active":"2024-09-17T03:14:55.332010+0000","last_became_peered":"2024-09-17T03:14:55.332010+0000","last_unstale":"2024-09-17T03:14:56.360502+0000","last_undegraded":"2024-09-17T03:14:56.360502+0000","last_fullsized":"2024-09-17T03:14:56.360502+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:19.084170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351935+0000","last_change":"2024-09-17T03:14:56.344117+0000","last_active":"2024-09-17T03:14:56.351935+0000","last_peered":"2024-09-17T03:14:56.351935+0000","last_clean":"2024-09-17T03:14:56.351935+0000","last_became_active":"2024-09-17T03:14:56.343821+0000","last_became_peered":"2024-09-17T03:14:56.343821+0000","last_unstale":"2024-09-17T03:14:56.351935+0000","last_undegraded":"2024-09-17T03:14:56.351935+0000","last_fullsized":"2024-09-17T03:14:56.351935+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:25.974302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.355844+0000","last_change":"2024-09-17T03:14:55.333495+0000","last_active":"2024-09-17T03:14:56.355844+0000","last_peered":"2024-09-17T03:14:56.355844+0000","last_clean":"2024-09-17T03:14:56.355844+0000","last_became_active":"2024-09-17T03:14:55.332779+0000","last_became_peered":"2024-09-17T03:14:55.332779+0000","last_unstale":"2024-09-17T03:14:56.355844+0000","last_undegraded":"2024-09-17T03:14:56.355844+0000","last_fullsized":"2024-09-17T03:14:56.355844+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:19.084170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T03:14:54.311860+0000","last_change":"2024-09-17T03:14:54.311860+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.311860+0000","last_undegraded":"2024-09-17T03:14:54.311860+0000","last_fullsized":"2024-09-17T03:14:54.311860+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.345568+0000","last_change":"2024-09-17T03:14:56.341404+0000","last_active":"2024-09-17T03:14:56.345568+0000","last_peered":"2024-09-17T03:14:56.345568+0000","last_clean":"2024-09-17T03:14:56.345568+0000","last_became_active":"2024-09-17T03:14:56.341086+0000","last_became_peered":"2024-09-17T03:14:56.341086+0000","last_unstale":"2024-09-17T03:14:56.345568+0000","last_undegraded":"2024-09-17T03:14:56.345568+0000","last_fullsized":"2024-09-17T03:14:56.345568+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:19:29.104716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T03:14:54.313012+0000","last_change":"2024-09-17T03:14:54.313012+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.313012+0000","last_undegraded":"2024-09-17T03:14:54.313012+0000","last_fullsized":"2024-09-17T03:14:54.313012+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351204+0000","last_change":"2024-09-17T03:14:56.347429+0000","last_active":"2024-09-17T03:14:56.351204+0000","last_peered":"2024-09-17T03:14:56.351204+0000","last_clean":"2024-09-17T03:14:56.351204+0000","last_became_active":"2024-09-17T03:14:56.347101+0000","last_became_peered":"2024-09-17T03:14:56.347101+0000","last_unstale":"2024-09-17T03:14:56.351204+0000","last_undegraded":"2024-09-17T03:14:56.351204+0000","last_fullsized":"2024-09-17T03:14:56.351204+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:46.793182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.340674+0000","last_change":"2024-09-17T03:14:55.330923+0000","last_active":"2024-09-17T03:14:57.340674+0000","last_peered":"2024-09-17T03:14:57.340674+0000","last_clean":"2024-09-17T03:14:57.340674+0000","last_became_active":"2024-09-17T03:14:55.330601+0000","last_became_peered":"2024-09-17T03:14:55.330601+0000","last_unstale":"2024-09-17T03:14:57.340674+0000","last_undegraded":"2024-09-17T03:14:57.340674+0000","last_fullsized":"2024-09-17T03:14:57.340674+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:21.326711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351107+0000","last_change":"2024-09-17T03:14:56.343137+0000","last_active":"2024-09-17T03:14:56.351107+0000","last_peered":"2024-09-17T03:14:56.351107+0000","last_clean":"2024-09-17T03:14:56.351107+0000","last_became_active":"2024-09-17T03:14:56.342838+0000","last_became_peered":"2024-09-17T03:14:56.342838+0000","last_unstale":"2024-09-17T03:14:56.351107+0000","last_undegraded":"2024-09-17T03:14:56.351107+0000","last_fullsized":"2024-09-17T03:14:56.351107+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:54:34.725205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341150+0000","last_change":"2024-09-17T03:14:55.331984+0000","last_active":"2024-09-17T03:14:57.341150+0000","last_peered":"2024-09-17T03:14:57.341150+0000","last_clean":"2024-09-17T03:14:57.341150+0000","last_became_active":"2024-09-17T03:14:55.331543+0000","last_became_peered":"2024-09-17T03:14:55.331543+0000","last_unstale":"2024-09-17T03:14:57.341150+0000","last_undegraded":"2024-09-17T03:14:57.341150+0000","last_fullsized":"2024-09-17T03:14:57.341150+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:06:37.330728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.338642+0000","last_change":"2024-09-17T03:14:56.336542+0000","last_active":"2024-09-17T03:14:56.338642+0000","last_peered":"2024-09-17T03:14:56.338642+0000","last_clean":"2024-09-17T03:14:56.338642+0000","last_became_active":"2024-09-17T03:14:56.336216+0000","last_became_peered":"2024-09-17T03:14:56.336216+0000","last_unstale":"2024-09-17T03:14:56.338642+0000","last_undegraded":"2024-09-17T03:14:56.338642+0000","last_fullsized":"2024-09-17T03:14:56.338642+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:10.905183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.355337+0000","last_change":"2024-09-17T03:14:55.335726+0000","last_active":"2024-09-17T03:14:56.355337+0000","last_peered":"2024-09-17T03:14:56.355337+0000","last_clean":"2024-09-17T03:14:56.355337+0000","last_became_active":"2024-09-17T03:14:55.335268+0000","last_became_peered":"2024-09-17T03:14:55.335268+0000","last_unstale":"2024-09-17T03:14:56.355337+0000","last_undegraded":"2024-09-17T03:14:56.355337+0000","last_fullsized":"2024-09-17T03:14:56.355337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:13.660225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344275+0000","last_change":"2024-09-17T03:14:56.336628+0000","last_active":"2024-09-17T03:14:57.344275+0000","last_peered":"2024-09-17T03:14:57.344275+0000","last_clean":"2024-09-17T03:14:57.344275+0000","last_became_active":"2024-09-17T03:14:56.336330+0000","last_became_peered":"2024-09-17T03:14:56.336330+0000","last_unstale":"2024-09-17T03:14:57.344275+0000","last_undegraded":"2024-09-17T03:14:57.344275+0000","last_fullsized":"2024-09-17T03:14:57.344275+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:02:13.758412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351038+0000","last_change":"2024-09-17T03:14:55.324166+0000","last_active":"2024-09-17T03:14:56.351038+0000","last_peered":"2024-09-17T03:14:56.351038+0000","last_clean":"2024-09-17T03:14:56.351038+0000","last_became_active":"2024-09-17T03:14:55.323844+0000","last_became_peered":"2024-09-17T03:14:55.323844+0000","last_unstale":"2024-09-17T03:14:56.351038+0000","last_undegraded":"2024-09-17T03:14:56.351038+0000","last_fullsized":"2024-09-17T03:14:56.351038+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:27:53.850399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341186+0000","last_change":"2024-09-17T03:14:56.337913+0000","last_active":"2024-09-17T03:14:57.341186+0000","last_peered":"2024-09-17T03:14:57.341186+0000","last_clean":"2024-09-17T03:14:57.341186+0000","last_became_active":"2024-09-17T03:14:56.337610+0000","last_became_peered":"2024-09-17T03:14:56.337610+0000","last_unstale":"2024-09-17T03:14:57.341186+0000","last_undegraded":"2024-09-17T03:14:57.341186+0000","last_fullsized":"2024-09-17T03:14:57.341186+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:08.606153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.325094+0000","last_change":"2024-09-17T03:14:55.333099+0000","last_active":"2024-09-17T03:14:56.325094+0000","last_peered":"2024-09-17T03:14:56.325094+0000","last_clean":"2024-09-17T03:14:56.325094+0000","last_became_active":"2024-09-17T03:14:55.332738+0000","last_became_peered":"2024-09-17T03:14:55.332738+0000","last_unstale":"2024-09-17T03:14:56.325094+0000","last_undegraded":"2024-09-17T03:14:56.325094+0000","last_fullsized":"2024-09-17T03:14:56.325094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:50.915622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.344808+0000","last_change":"2024-09-17T03:14:56.340119+0000","last_active":"2024-09-17T03:14:56.344808+0000","last_peered":"2024-09-17T03:14:56.344808+0000","last_clean":"2024-09-17T03:14:56.344808+0000","last_became_active":"2024-09-17T03:14:56.339795+0000","last_became_peered":"2024-09-17T03:14:56.339795+0000","last_unstale":"2024-09-17T03:14:56.344808+0000","last_undegraded":"2024-09-17T03:14:56.344808+0000","last_fullsized":"2024-09-17T03:14:56.344808+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:04.002930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344218+0000","last_change":"2024-09-17T03:14:55.327065+0000","last_active":"2024-09-17T03:14:57.344218+0000","last_peered":"2024-09-17T03:14:57.344218+0000","last_clean":"2024-09-17T03:14:57.344218+0000","last_became_active":"2024-09-17T03:14:55.326774+0000","last_became_peered":"2024-09-17T03:14:55.326774+0000","last_unstale":"2024-09-17T03:14:57.344218+0000","last_undegraded":"2024-09-17T03:14:57.344218+0000","last_fullsized":"2024-09-17T03:14:57.344218+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:37.138244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344009+0000","last_change":"2024-09-17T03:14:56.335950+0000","last_active":"2024-09-17T03:14:57.344009+0000","last_peered":"2024-09-17T03:14:57.344009+0000","last_clean":"2024-09-17T03:14:57.344009+0000","last_became_active":"2024-09-17T03:14:56.335641+0000","last_became_peered":"2024-09-17T03:14:56.335641+0000","last_unstale":"2024-09-17T03:14:57.344009+0000","last_undegraded":"2024-09-17T03:14:57.344009+0000","last_fullsized":"2024-09-17T03:14:57.344009+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:03.609971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.329227+0000","last_change":"2024-09-17T03:14:55.335625+0000","last_active":"2024-09-17T03:14:56.329227+0000","last_peered":"2024-09-17T03:14:56.329227+0000","last_clean":"2024-09-17T03:14:56.329227+0000","last_became_active":"2024-09-17T03:14:55.335317+0000","last_became_peered":"2024-09-17T03:14:55.335317+0000","last_unstale":"2024-09-17T03:14:56.329227+0000","last_undegraded":"2024-09-17T03:14:56.329227+0000","last_fullsized":"2024-09-17T03:14:56.329227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:47.254999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.357401+0000","last_change":"2024-09-17T03:14:56.346898+0000","last_active":"2024-09-17T03:14:56.357401+0000","last_peered":"2024-09-17T03:14:56.357401+0000","last_clean":"2024-09-17T03:14:56.357401+0000","last_became_active":"2024-09-17T03:14:56.346603+0000","last_became_peered":"2024-09-17T03:14:56.346603+0000","last_unstale":"2024-09-17T03:14:56.357401+0000","last_undegraded":"2024-09-17T03:14:56.357401+0000","last_fullsized":"2024-09-17T03:14:56.357401+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:26:13.257738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.328281+0000","last_change":"2024-09-17T03:14:55.328041+0000","last_active":"2024-09-17T03:14:56.328281+0000","last_peered":"2024-09-17T03:14:56.328281+0000","last_clean":"2024-09-17T03:14:56.328281+0000","last_became_active":"2024-09-17T03:14:55.327533+0000","last_became_peered":"2024-09-17T03:14:55.327533+0000","last_unstale":"2024-09-17T03:14:56.328281+0000","last_undegraded":"2024-09-17T03:14:56.328281+0000","last_fullsized":"2024-09-17T03:14:56.328281+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:21.326711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.337735+0000","last_change":"2024-09-17T03:14:56.336599+0000","last_active":"2024-09-17T03:14:56.337735+0000","last_peered":"2024-09-17T03:14:56.337735+0000","last_clean":"2024-09-17T03:14:56.337735+0000","last_became_active":"2024-09-17T03:14:56.336243+0000","last_became_peered":"2024-09-17T03:14:56.336243+0000","last_unstale":"2024-09-17T03:14:56.337735+0000","last_undegraded":"2024-09-17T03:14:56.337735+0000","last_fullsized":"2024-09-17T03:14:56.337735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:06:38.336520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.329616+0000","last_change":"2024-09-17T03:14:55.324984+0000","last_active":"2024-09-17T03:14:56.329616+0000","last_peered":"2024-09-17T03:14:56.329616+0000","last_clean":"2024-09-17T03:14:56.329616+0000","last_became_active":"2024-09-17T03:14:55.324696+0000","last_became_peered":"2024-09-17T03:14:55.324696+0000","last_unstale":"2024-09-17T03:14:56.329616+0000","last_undegraded":"2024-09-17T03:14:56.329616+0000","last_fullsized":"2024-09-17T03:14:56.329616+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:10.371792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341211+0000","last_change":"2024-09-17T03:14:56.338902+0000","last_active":"2024-09-17T03:14:57.341211+0000","last_peered":"2024-09-17T03:14:57.341211+0000","last_clean":"2024-09-17T03:14:57.341211+0000","last_became_active":"2024-09-17T03:14:56.338504+0000","last_became_peered":"2024-09-17T03:14:56.338504+0000","last_unstale":"2024-09-17T03:14:57.341211+0000","last_undegraded":"2024-09-17T03:14:57.341211+0000","last_fullsized":"2024-09-17T03:14:57.341211+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:16.330515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.357201+0000","last_change":"2024-09-17T03:14:55.328618+0000","last_active":"2024-09-17T03:14:56.357201+0000","last_peered":"2024-09-17T03:14:56.357201+0000","last_clean":"2024-09-17T03:14:56.357201+0000","last_became_active":"2024-09-17T03:14:55.328321+0000","last_became_peered":"2024-09-17T03:14:55.328321+0000","last_unstale":"2024-09-17T03:14:56.357201+0000","last_undegraded":"2024-09-17T03:14:56.357201+0000","last_fullsized":"2024-09-17T03:14:56.357201+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:30:39.180289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.354677+0000","last_change":"2024-09-17T03:14:56.341818+0000","last_active":"2024-09-17T03:14:56.354677+0000","last_peered":"2024-09-17T03:14:56.354677+0000","last_clean":"2024-09-17T03:14:56.354677+0000","last_became_active":"2024-09-17T03:14:56.341526+0000","last_became_peered":"2024-09-17T03:14:56.341526+0000","last_unstale":"2024-09-17T03:14:56.354677+0000","last_undegraded":"2024-09-17T03:14:56.354677+0000","last_fullsized":"2024-09-17T03:14:56.354677+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:35.033454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344149+0000","last_change":"2024-09-17T03:14:55.325076+0000","last_active":"2024-09-17T03:14:57.344149+0000","last_peered":"2024-09-17T03:14:57.344149+0000","last_clean":"2024-09-17T03:14:57.344149+0000","last_became_active":"2024-09-17T03:14:55.324738+0000","last_became_peered":"2024-09-17T03:14:55.324738+0000","last_unstale":"2024-09-17T03:14:57.344149+0000","last_undegraded":"2024-09-17T03:14:57.344149+0000","last_fullsized":"2024-09-17T03:14:57.344149+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:26:33.226093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341372+0000","last_change":"2024-09-17T03:14:56.348451+0000","last_active":"2024-09-17T03:14:57.341372+0000","last_peered":"2024-09-17T03:14:57.341372+0000","last_clean":"2024-09-17T03:14:57.341372+0000","last_became_active":"2024-09-17T03:14:56.348169+0000","last_became_peered":"2024-09-17T03:14:56.348169+0000","last_unstale":"2024-09-17T03:14:57.341372+0000","last_undegraded":"2024-09-17T03:14:57.341372+0000","last_fullsized":"2024-09-17T03:14:57.341372+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:47.114511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.332977+0000","last_change":"2024-09-17T03:14:55.327511+0000","last_active":"2024-09-17T03:14:56.332977+0000","last_peered":"2024-09-17T03:14:56.332977+0000","last_clean":"2024-09-17T03:14:56.332977+0000","last_became_active":"2024-09-17T03:14:55.327195+0000","last_became_peered":"2024-09-17T03:14:55.327195+0000","last_unstale":"2024-09-17T03:14:56.332977+0000","last_undegraded":"2024-09-17T03:14:56.332977+0000","last_fullsized":"2024-09-17T03:14:56.332977+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:13.660225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.344622+0000","last_change":"2024-09-17T03:14:56.337722+0000","last_active":"2024-09-17T03:14:56.344622+0000","last_peered":"2024-09-17T03:14:56.344622+0000","last_clean":"2024-09-17T03:14:56.344622+0000","last_became_active":"2024-09-17T03:14:56.337353+0000","last_became_peered":"2024-09-17T03:14:56.337353+0000","last_unstale":"2024-09-17T03:14:56.344622+0000","last_undegraded":"2024-09-17T03:14:56.344622+0000","last_fullsized":"2024-09-17T03:14:56.344622+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:34:03.584090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.337538+0000","last_change":"2024-09-17T03:14:55.328438+0000","last_active":"2024-09-17T03:14:56.337538+0000","last_peered":"2024-09-17T03:14:56.337538+0000","last_clean":"2024-09-17T03:14:56.337538+0000","last_became_active":"2024-09-17T03:14:55.328158+0000","last_became_peered":"2024-09-17T03:14:55.328158+0000","last_unstale":"2024-09-17T03:14:56.337538+0000","last_undegraded":"2024-09-17T03:14:56.337538+0000","last_fullsized":"2024-09-17T03:14:56.337538+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:19:28.098924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.331672+0000","last_change":"2024-09-17T03:14:55.330495+0000","last_active":"2024-09-17T03:14:56.331672+0000","last_peered":"2024-09-17T03:14:56.331672+0000","last_clean":"2024-09-17T03:14:56.331672+0000","last_became_active":"2024-09-17T03:14:55.330159+0000","last_became_peered":"2024-09-17T03:14:55.330159+0000","last_unstale":"2024-09-17T03:14:56.331672+0000","last_undegraded":"2024-09-17T03:14:56.331672+0000","last_fullsized":"2024-09-17T03:14:56.331672+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:55.212014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.331656+0000","last_change":"2024-09-17T03:14:55.331670+0000","last_active":"2024-09-17T03:14:56.331656+0000","last_peered":"2024-09-17T03:14:56.331656+0000","last_clean":"2024-09-17T03:14:56.331656+0000","last_became_active":"2024-09-17T03:14:55.331359+0000","last_became_peered":"2024-09-17T03:14:55.331359+0000","last_unstale":"2024-09-17T03:14:56.331656+0000","last_undegraded":"2024-09-17T03:14:56.331656+0000","last_fullsized":"2024-09-17T03:14:56.331656+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:55.212014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351181+0000","last_change":"2024-09-17T03:14:56.341402+0000","last_active":"2024-09-17T03:14:56.351181+0000","last_peered":"2024-09-17T03:14:56.351181+0000","last_clean":"2024-09-17T03:14:56.351181+0000","last_became_active":"2024-09-17T03:14:56.341095+0000","last_became_peered":"2024-09-17T03:14:56.341095+0000","last_unstale":"2024-09-17T03:14:56.351181+0000","last_undegraded":"2024-09-17T03:14:56.351181+0000","last_fullsized":"2024-09-17T03:14:56.351181+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:33:09.441087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T03:14:54.312546+0000","last_change":"2024-09-17T03:14:54.312546+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.312546+0000","last_undegraded":"2024-09-17T03:14:54.312546+0000","last_fullsized":"2024-09-17T03:14:54.312546+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.343160+0000","last_change":"2024-09-17T03:14:56.337604+0000","last_active":"2024-09-17T03:14:56.343160+0000","last_peered":"2024-09-17T03:14:56.343160+0000","last_clean":"2024-09-17T03:14:56.343160+0000","last_became_active":"2024-09-17T03:14:56.337283+0000","last_became_peered":"2024-09-17T03:14:56.337283+0000","last_unstale":"2024-09-17T03:14:56.343160+0000","last_undegraded":"2024-09-17T03:14:56.343160+0000","last_fullsized":"2024-09-17T03:14:56.343160+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:17.535415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.333475+0000","last_change":"2024-09-17T03:14:55.322959+0000","last_active":"2024-09-17T03:14:56.333475+0000","last_peered":"2024-09-17T03:14:56.333475+0000","last_clean":"2024-09-17T03:14:56.333475+0000","last_became_active":"2024-09-17T03:14:55.322475+0000","last_became_peered":"2024-09-17T03:14:55.322475+0000","last_unstale":"2024-09-17T03:14:56.333475+0000","last_undegraded":"2024-09-17T03:14:56.333475+0000","last_fullsized":"2024-09-17T03:14:56.333475+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:57.870235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":127,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.328846+0000","last_change":"2024-09-17T03:14:04.862167+0000","last_active":"2024-09-17T03:14:56.328846+0000","last_peered":"2024-09-17T03:14:56.328846+0000","last_clean":"2024-09-17T03:14:56.328846+0000","last_became_active":"2024-09-17T03:14:04.861731+0000","last_became_peered":"2024-09-17T03:14:04.861731+0000","last_unstale":"2024-09-17T03:14:56.328846+0000","last_undegraded":"2024-09-17T03:14:56.328846+0000","last_fullsized":"2024-09-17T03:14:56.328846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_clean_scrub_stamp":"2024-09-17T03:14:03.847721+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:32.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":104,"acting":104,"num_store_stats":7},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509456,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":368640,"data_stored":139029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":376832,"data_stored":140619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509455,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":962560,"data_stored":729957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":380928,"data_stored":139783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":368640,"data_stored":139041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":962560,"data_stored":729445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":368640,"data_stored":139097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":327680,"data_stored":120140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:14:59.686 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:14:59.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.836+0000 7fcce59a1640 1 -- 172.21.15.87:0/2913378871 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcce00b0010 msgr2=0x7fcce00b0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:59.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.836+0000 7fcce59a1640 1 --2- 172.21.15.87:0/2913378871 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcce00b0010 0x7fcce00b0420 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fccd0005fd0 tx=0x7fccd002a530 comp rx=0 tx=0).stop 2024-09-17T03:14:59.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.836+0000 7fcce59a1640 1 -- 172.21.15.87:0/2913378871 shutdown_connections 2024-09-17T03:14:59.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.836+0000 7fcce59a1640 1 --2- 172.21.15.87:0/2913378871 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcce00b0010 0x7fcce00b0420 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.836+0000 7fcce59a1640 1 --2- 172.21.15.87:0/2913378871 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcce00d04e0 0x7fcce015f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.836+0000 7fcce59a1640 1 -- 172.21.15.87:0/2913378871 >> 172.21.15.87:0/2913378871 conn(0x7fcce00d1110 msgr2=0x7fcce00d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:14:59.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.836+0000 7fcce59a1640 1 -- 172.21.15.87:0/2913378871 shutdown_connections 2024-09-17T03:14:59.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.836+0000 7fcce59a1640 1 -- 172.21.15.87:0/2913378871 wait complete. 2024-09-17T03:14:59.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.838+0000 7fcce59a1640 1 Processor -- start 2024-09-17T03:14:59.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.838+0000 7fcce59a1640 1 -- start start 2024-09-17T03:14:59.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.839+0000 7fcce59a1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcce00b0010 0x7fcce014b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:59.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.839+0000 7fcce59a1640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcce00d04e0 0x7fcce00ae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:59.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.839+0000 7fcce59a1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcce015b490 0x7fcce00aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:59.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.839+0000 7fcce59a1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcce00cc830 con 0x7fcce00b0010 2024-09-17T03:14:59.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.839+0000 7fcce59a1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcce00cc6b0 con 0x7fcce015b490 2024-09-17T03:14:59.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.839+0000 7fcce59a1640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcce00cc9b0 con 0x7fcce00d04e0 2024-09-17T03:14:59.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.839+0000 7fccdeffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcce00b0010 0x7fcce014b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:59.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.839+0000 7fccdeffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcce00b0010 0x7fcce014b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40024/0 (socket says 172.21.15.87:40024) 2024-09-17T03:14:59.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.839+0000 7fccdeffd640 1 -- 172.21.15.87:0/2547986350 learned_addr learned my addr 172.21.15.87:0/2547986350 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:14:59.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.839+0000 7fccde7fc640 1 --2- 172.21.15.87:0/2547986350 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcce00d04e0 0x7fcce00ae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:59.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.839+0000 7fccdeffd640 1 -- 172.21.15.87:0/2547986350 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcce00d04e0 msgr2=0x7fcce00ae7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:59.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.839+0000 7fccdf7fe640 1 --2- 172.21.15.87:0/2547986350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcce015b490 0x7fcce00aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:59.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.839+0000 7fccdeffd640 1 --2- 172.21.15.87:0/2547986350 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcce00d04e0 0x7fcce00ae7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.839+0000 7fccdeffd640 1 -- 172.21.15.87:0/2547986350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcce015b490 msgr2=0x7fcce00aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:14:59.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.839+0000 7fccdeffd640 1 --2- 172.21.15.87:0/2547986350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcce015b490 0x7fcce00aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:14:59.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.839+0000 7fccdeffd640 1 -- 172.21.15.87:0/2547986350 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce014b6f0 con 0x7fcce00b0010 2024-09-17T03:14:59.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.839+0000 7fccde7fc640 1 --2- 172.21.15.87:0/2547986350 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcce00d04e0 0x7fcce00ae7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:14:59.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.839+0000 7fccdf7fe640 1 --2- 172.21.15.87:0/2547986350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcce015b490 0x7fcce00aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:14:59.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.840+0000 7fccdeffd640 1 --2- 172.21.15.87:0/2547986350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcce00b0010 0x7fcce014b1b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fccd800b440 tx=0x7fccd800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:59.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.840+0000 7fccc7fff640 1 -- 172.21.15.87:0/2547986350 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd8015020 con 0x7fcce00b0010 2024-09-17T03:14:59.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.840+0000 7fccc7fff640 1 -- 172.21.15.87:0/2547986350 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fccd8011980 con 0x7fcce00b0010 2024-09-17T03:14:59.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.840+0000 7fcce59a1640 1 -- 172.21.15.87:0/2547986350 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce014bc00 con 0x7fcce00b0010 2024-09-17T03:14:59.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.840+0000 7fccc7fff640 1 -- 172.21.15.87:0/2547986350 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccd8011dd0 con 0x7fcce00b0010 2024-09-17T03:14:59.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.840+0000 7fcce59a1640 1 -- 172.21.15.87:0/2547986350 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce01492a0 con 0x7fcce00b0010 2024-09-17T03:14:59.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.841+0000 7fcce59a1640 1 -- 172.21.15.87:0/2547986350 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcce00b0420 con 0x7fcce00b0010 2024-09-17T03:14:59.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.842+0000 7fccc7fff640 1 -- 172.21.15.87:0/2547986350 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fccd8018020 con 0x7fcce00b0010 2024-09-17T03:14:59.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.842+0000 7fccc7fff640 1 --2- 172.21.15.87:0/2547986350 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcccc078b90 0x7fcccc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:14:59.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.842+0000 7fccc7fff640 1 -- 172.21.15.87:0/2547986350 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fccd809adb0 con 0x7fcce00b0010 2024-09-17T03:14:59.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.843+0000 7fccde7fc640 1 --2- 172.21.15.87:0/2547986350 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcccc078b90 0x7fcccc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:14:59.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.843+0000 7fccde7fc640 1 --2- 172.21.15.87:0/2547986350 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcccc078b90 0x7fcccc07b0a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fccd4007eb0 tx=0x7fccd40073c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:14:59.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:14:59.847+0000 7fccc7fff640 1 -- 172.21.15.87:0/2547986350 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fccd801f020 con 0x7fcce00b0010 2024-09-17T03:15:00.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.018+0000 7fcce59a1640 1 -- 172.21.15.87:0/2547986350 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fcce00b7090 con 0x7fcccc078b90 2024-09-17T03:15:00.021 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.023+0000 7fccc7fff640 1 -- 172.21.15.87:0/2547986350 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249913 (secure 0 0 0) 0x7fcce00b7090 con 0x7fcccc078b90 2024-09-17T03:15:00.024 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:00.025 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-09-17T03:15:00.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.026+0000 7fcce59a1640 1 -- 172.21.15.87:0/2547986350 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcccc078b90 msgr2=0x7fcccc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:00.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.026+0000 7fcce59a1640 1 --2- 172.21.15.87:0/2547986350 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcccc078b90 0x7fcccc07b0a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fccd4007eb0 tx=0x7fccd40073c0 comp rx=0 tx=0).stop 2024-09-17T03:15:00.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.026+0000 7fcce59a1640 1 -- 172.21.15.87:0/2547986350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcce00b0010 msgr2=0x7fcce014b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:00.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.026+0000 7fcce59a1640 1 --2- 172.21.15.87:0/2547986350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcce00b0010 0x7fcce014b1b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fccd800b440 tx=0x7fccd800b900 comp rx=0 tx=0).stop 2024-09-17T03:15:00.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.026+0000 7fcce59a1640 1 -- 172.21.15.87:0/2547986350 shutdown_connections 2024-09-17T03:15:00.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.026+0000 7fcce59a1640 1 --2- 172.21.15.87:0/2547986350 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcccc078b90 0x7fcccc07b0a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:00.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.026+0000 7fcce59a1640 1 --2- 172.21.15.87:0/2547986350 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcce015b490 0x7fcce00aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:00.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.026+0000 7fcce59a1640 1 --2- 172.21.15.87:0/2547986350 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcce00d04e0 0x7fcce00ae7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:00.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.026+0000 7fcce59a1640 1 --2- 172.21.15.87:0/2547986350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcce00b0010 0x7fcce014b1b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:00.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.026+0000 7fcce59a1640 1 -- 172.21.15.87:0/2547986350 >> 172.21.15.87:0/2547986350 conn(0x7fcce00d1110 msgr2=0x7fcce00d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:00.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.026+0000 7fcce59a1640 1 -- 172.21.15.87:0/2547986350 shutdown_connections 2024-09-17T03:15:00.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.026+0000 7fcce59a1640 1 -- 172.21.15.87:0/2547986350 wait complete. 2024-09-17T03:15:00.036 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-09-17T03:14:59.427221+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":142,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":170,"acting":170,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":94,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224464,"kb_used_data":4020,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750256,"statfs":{"total":773094113280,"available":772864262144,"internally_reserved":0,"allocated":4116480,"data_stored":2277111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.117237"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351505+0000","last_change":"2024-09-17T03:14:56.344756+0000","last_active":"2024-09-17T03:14:56.351505+0000","last_peered":"2024-09-17T03:14:56.351505+0000","last_clean":"2024-09-17T03:14:56.351505+0000","last_became_active":"2024-09-17T03:14:56.343497+0000","last_became_peered":"2024-09-17T03:14:56.343497+0000","last_unstale":"2024-09-17T03:14:56.351505+0000","last_undegraded":"2024-09-17T03:14:56.351505+0000","last_fullsized":"2024-09-17T03:14:56.351505+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:07.377313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344120+0000","last_change":"2024-09-17T03:14:56.348503+0000","last_active":"2024-09-17T03:14:57.344120+0000","last_peered":"2024-09-17T03:14:57.344120+0000","last_clean":"2024-09-17T03:14:57.344120+0000","last_became_active":"2024-09-17T03:14:56.348172+0000","last_became_peered":"2024-09-17T03:14:56.348172+0000","last_unstale":"2024-09-17T03:14:57.344120+0000","last_undegraded":"2024-09-17T03:14:57.344120+0000","last_fullsized":"2024-09-17T03:14:57.344120+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:52:09.125634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.349842+0000","last_change":"2024-09-17T03:14:56.347013+0000","last_active":"2024-09-17T03:14:56.349842+0000","last_peered":"2024-09-17T03:14:56.349842+0000","last_clean":"2024-09-17T03:14:56.349842+0000","last_became_active":"2024-09-17T03:14:56.346718+0000","last_became_peered":"2024-09-17T03:14:56.346718+0000","last_unstale":"2024-09-17T03:14:56.349842+0000","last_undegraded":"2024-09-17T03:14:56.349842+0000","last_fullsized":"2024-09-17T03:14:56.349842+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:17.382050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.343763+0000","last_change":"2024-09-17T03:14:56.344528+0000","last_active":"2024-09-17T03:14:57.343763+0000","last_peered":"2024-09-17T03:14:57.343763+0000","last_clean":"2024-09-17T03:14:57.343763+0000","last_became_active":"2024-09-17T03:14:56.344226+0000","last_became_peered":"2024-09-17T03:14:56.344226+0000","last_unstale":"2024-09-17T03:14:57.343763+0000","last_undegraded":"2024-09-17T03:14:57.343763+0000","last_fullsized":"2024-09-17T03:14:57.343763+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:09:48.039023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.340976+0000","last_change":"2024-09-17T03:14:56.342382+0000","last_active":"2024-09-17T03:14:57.340976+0000","last_peered":"2024-09-17T03:14:57.340976+0000","last_clean":"2024-09-17T03:14:57.340976+0000","last_became_active":"2024-09-17T03:14:56.341980+0000","last_became_peered":"2024-09-17T03:14:56.341980+0000","last_unstale":"2024-09-17T03:14:57.340976+0000","last_undegraded":"2024-09-17T03:14:57.340976+0000","last_fullsized":"2024-09-17T03:14:57.340976+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:11.714679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.345446+0000","last_change":"2024-09-17T03:14:56.343019+0000","last_active":"2024-09-17T03:14:56.345446+0000","last_peered":"2024-09-17T03:14:56.345446+0000","last_clean":"2024-09-17T03:14:56.345446+0000","last_became_active":"2024-09-17T03:14:56.342705+0000","last_became_peered":"2024-09-17T03:14:56.342705+0000","last_unstale":"2024-09-17T03:14:56.345446+0000","last_undegraded":"2024-09-17T03:14:56.345446+0000","last_fullsized":"2024-09-17T03:14:56.345446+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:04.002930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.352770+0000","last_change":"2024-09-17T03:14:56.346692+0000","last_active":"2024-09-17T03:14:56.352770+0000","last_peered":"2024-09-17T03:14:56.352770+0000","last_clean":"2024-09-17T03:14:56.352770+0000","last_became_active":"2024-09-17T03:14:56.346397+0000","last_became_peered":"2024-09-17T03:14:56.346397+0000","last_unstale":"2024-09-17T03:14:56.352770+0000","last_undegraded":"2024-09-17T03:14:56.352770+0000","last_fullsized":"2024-09-17T03:14:56.352770+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:28.743694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.344785+0000","last_change":"2024-09-17T03:14:56.339990+0000","last_active":"2024-09-17T03:14:56.344785+0000","last_peered":"2024-09-17T03:14:56.344785+0000","last_clean":"2024-09-17T03:14:56.344785+0000","last_became_active":"2024-09-17T03:14:56.339281+0000","last_became_peered":"2024-09-17T03:14:56.339281+0000","last_unstale":"2024-09-17T03:14:56.344785+0000","last_undegraded":"2024-09-17T03:14:56.344785+0000","last_fullsized":"2024-09-17T03:14:56.344785+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:32:19.770373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351104+0000","last_change":"2024-09-17T03:14:56.342266+0000","last_active":"2024-09-17T03:14:56.351104+0000","last_peered":"2024-09-17T03:14:56.351104+0000","last_clean":"2024-09-17T03:14:56.351104+0000","last_became_active":"2024-09-17T03:14:56.341978+0000","last_became_peered":"2024-09-17T03:14:56.341978+0000","last_unstale":"2024-09-17T03:14:56.351104+0000","last_undegraded":"2024-09-17T03:14:56.351104+0000","last_fullsized":"2024-09-17T03:14:56.351104+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:50:28.343075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.345419+0000","last_change":"2024-09-17T03:14:56.342630+0000","last_active":"2024-09-17T03:14:56.345419+0000","last_peered":"2024-09-17T03:14:56.345419+0000","last_clean":"2024-09-17T03:14:56.345419+0000","last_became_active":"2024-09-17T03:14:56.342175+0000","last_became_peered":"2024-09-17T03:14:56.342175+0000","last_unstale":"2024-09-17T03:14:56.345419+0000","last_undegraded":"2024-09-17T03:14:56.345419+0000","last_fullsized":"2024-09-17T03:14:56.345419+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:02:14.647806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.357637+0000","last_change":"2024-09-17T03:14:56.351892+0000","last_active":"2024-09-17T03:14:56.357637+0000","last_peered":"2024-09-17T03:14:56.357637+0000","last_clean":"2024-09-17T03:14:56.357637+0000","last_became_active":"2024-09-17T03:14:56.351149+0000","last_became_peered":"2024-09-17T03:14:56.351149+0000","last_unstale":"2024-09-17T03:14:56.357637+0000","last_undegraded":"2024-09-17T03:14:56.357637+0000","last_fullsized":"2024-09-17T03:14:56.357637+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:14:16.498828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351551+0000","last_change":"2024-09-17T03:14:56.349422+0000","last_active":"2024-09-17T03:14:56.351551+0000","last_peered":"2024-09-17T03:14:56.351551+0000","last_clean":"2024-09-17T03:14:56.351551+0000","last_became_active":"2024-09-17T03:14:56.349132+0000","last_became_peered":"2024-09-17T03:14:56.349132+0000","last_unstale":"2024-09-17T03:14:56.351551+0000","last_undegraded":"2024-09-17T03:14:56.351551+0000","last_fullsized":"2024-09-17T03:14:56.351551+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:27.164191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.340907+0000","last_change":"2024-09-17T03:14:56.337002+0000","last_active":"2024-09-17T03:14:56.340907+0000","last_peered":"2024-09-17T03:14:56.340907+0000","last_clean":"2024-09-17T03:14:56.340907+0000","last_became_active":"2024-09-17T03:14:56.336649+0000","last_became_peered":"2024-09-17T03:14:56.336649+0000","last_unstale":"2024-09-17T03:14:56.340907+0000","last_undegraded":"2024-09-17T03:14:56.340907+0000","last_fullsized":"2024-09-17T03:14:56.340907+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:02:48.048417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.354704+0000","last_change":"2024-09-17T03:14:56.346405+0000","last_active":"2024-09-17T03:14:56.354704+0000","last_peered":"2024-09-17T03:14:56.354704+0000","last_clean":"2024-09-17T03:14:56.354704+0000","last_became_active":"2024-09-17T03:14:56.346037+0000","last_became_peered":"2024-09-17T03:14:56.346037+0000","last_unstale":"2024-09-17T03:14:56.354704+0000","last_undegraded":"2024-09-17T03:14:56.354704+0000","last_fullsized":"2024-09-17T03:14:56.354704+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:04:18.618725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341208+0000","last_change":"2024-09-17T03:14:56.345147+0000","last_active":"2024-09-17T03:14:57.341208+0000","last_peered":"2024-09-17T03:14:57.341208+0000","last_clean":"2024-09-17T03:14:57.341208+0000","last_became_active":"2024-09-17T03:14:56.344844+0000","last_became_peered":"2024-09-17T03:14:56.344844+0000","last_unstale":"2024-09-17T03:14:57.341208+0000","last_undegraded":"2024-09-17T03:14:57.341208+0000","last_fullsized":"2024-09-17T03:14:57.341208+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:30:40.186081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341134+0000","last_change":"2024-09-17T03:14:56.339650+0000","last_active":"2024-09-17T03:14:57.341134+0000","last_peered":"2024-09-17T03:14:57.341134+0000","last_clean":"2024-09-17T03:14:57.341134+0000","last_became_active":"2024-09-17T03:14:56.338719+0000","last_became_peered":"2024-09-17T03:14:56.338719+0000","last_unstale":"2024-09-17T03:14:57.341134+0000","last_undegraded":"2024-09-17T03:14:57.341134+0000","last_fullsized":"2024-09-17T03:14:57.341134+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:51:33.167537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351809+0000","last_change":"2024-09-17T03:14:56.341270+0000","last_active":"2024-09-17T03:14:56.351809+0000","last_peered":"2024-09-17T03:14:56.351809+0000","last_clean":"2024-09-17T03:14:56.351809+0000","last_became_active":"2024-09-17T03:14:56.340876+0000","last_became_peered":"2024-09-17T03:14:56.340876+0000","last_unstale":"2024-09-17T03:14:56.351809+0000","last_undegraded":"2024-09-17T03:14:56.351809+0000","last_fullsized":"2024-09-17T03:14:56.351809+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:07.377313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.349912+0000","last_change":"2024-09-17T03:14:56.344410+0000","last_active":"2024-09-17T03:14:56.349912+0000","last_peered":"2024-09-17T03:14:56.349912+0000","last_clean":"2024-09-17T03:14:56.349912+0000","last_became_active":"2024-09-17T03:14:56.344151+0000","last_became_peered":"2024-09-17T03:14:56.344151+0000","last_unstale":"2024-09-17T03:14:56.349912+0000","last_undegraded":"2024-09-17T03:14:56.349912+0000","last_fullsized":"2024-09-17T03:14:56.349912+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:38:12.850097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.356153+0000","last_change":"2024-09-17T03:14:56.354343+0000","last_active":"2024-09-17T03:14:56.356153+0000","last_peered":"2024-09-17T03:14:56.356153+0000","last_clean":"2024-09-17T03:14:56.356153+0000","last_became_active":"2024-09-17T03:14:56.354020+0000","last_became_peered":"2024-09-17T03:14:56.354020+0000","last_unstale":"2024-09-17T03:14:56.356153+0000","last_undegraded":"2024-09-17T03:14:56.356153+0000","last_fullsized":"2024-09-17T03:14:56.356153+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:23.062711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351758+0000","last_change":"2024-09-17T03:14:56.340466+0000","last_active":"2024-09-17T03:14:56.351758+0000","last_peered":"2024-09-17T03:14:56.351758+0000","last_clean":"2024-09-17T03:14:56.351758+0000","last_became_active":"2024-09-17T03:14:56.340135+0000","last_became_peered":"2024-09-17T03:14:56.340135+0000","last_unstale":"2024-09-17T03:14:56.351758+0000","last_undegraded":"2024-09-17T03:14:56.351758+0000","last_fullsized":"2024-09-17T03:14:56.351758+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:05.079987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.356113+0000","last_change":"2024-09-17T03:14:56.353500+0000","last_active":"2024-09-17T03:14:56.356113+0000","last_peered":"2024-09-17T03:14:56.356113+0000","last_clean":"2024-09-17T03:14:56.356113+0000","last_became_active":"2024-09-17T03:14:56.353211+0000","last_became_peered":"2024-09-17T03:14:56.353211+0000","last_unstale":"2024-09-17T03:14:56.356113+0000","last_undegraded":"2024-09-17T03:14:56.356113+0000","last_fullsized":"2024-09-17T03:14:56.356113+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:11.473585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.343839+0000","last_change":"2024-09-17T03:14:56.329704+0000","last_active":"2024-09-17T03:14:57.343839+0000","last_peered":"2024-09-17T03:14:57.343839+0000","last_clean":"2024-09-17T03:14:57.343839+0000","last_became_active":"2024-09-17T03:14:56.329354+0000","last_became_peered":"2024-09-17T03:14:56.329354+0000","last_unstale":"2024-09-17T03:14:57.343839+0000","last_undegraded":"2024-09-17T03:14:57.343839+0000","last_fullsized":"2024-09-17T03:14:57.343839+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:59:50.923956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.352937+0000","last_change":"2024-09-17T03:14:56.348309+0000","last_active":"2024-09-17T03:14:56.352937+0000","last_peered":"2024-09-17T03:14:56.352937+0000","last_clean":"2024-09-17T03:14:56.352937+0000","last_became_active":"2024-09-17T03:14:56.347990+0000","last_became_peered":"2024-09-17T03:14:56.347990+0000","last_unstale":"2024-09-17T03:14:56.352937+0000","last_undegraded":"2024-09-17T03:14:56.352937+0000","last_fullsized":"2024-09-17T03:14:56.352937+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:25.974302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.348883+0000","last_change":"2024-09-17T03:14:56.342697+0000","last_active":"2024-09-17T03:14:56.348883+0000","last_peered":"2024-09-17T03:14:56.348883+0000","last_clean":"2024-09-17T03:14:56.348883+0000","last_became_active":"2024-09-17T03:14:56.342375+0000","last_became_peered":"2024-09-17T03:14:56.342375+0000","last_unstale":"2024-09-17T03:14:56.348883+0000","last_undegraded":"2024-09-17T03:14:56.348883+0000","last_fullsized":"2024-09-17T03:14:56.348883+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:04.426421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.350990+0000","last_change":"2024-09-17T03:14:56.346711+0000","last_active":"2024-09-17T03:14:56.350990+0000","last_peered":"2024-09-17T03:14:56.350990+0000","last_clean":"2024-09-17T03:14:56.350990+0000","last_became_active":"2024-09-17T03:14:56.346399+0000","last_became_peered":"2024-09-17T03:14:56.346399+0000","last_unstale":"2024-09-17T03:14:56.350990+0000","last_undegraded":"2024-09-17T03:14:56.350990+0000","last_fullsized":"2024-09-17T03:14:56.350990+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:20.900685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.352810+0000","last_change":"2024-09-17T03:14:56.348432+0000","last_active":"2024-09-17T03:14:56.352810+0000","last_peered":"2024-09-17T03:14:56.352810+0000","last_clean":"2024-09-17T03:14:56.352810+0000","last_became_active":"2024-09-17T03:14:56.348048+0000","last_became_peered":"2024-09-17T03:14:56.348048+0000","last_unstale":"2024-09-17T03:14:56.352810+0000","last_undegraded":"2024-09-17T03:14:56.352810+0000","last_fullsized":"2024-09-17T03:14:56.352810+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:26:13.257738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351063+0000","last_change":"2024-09-17T03:14:56.346523+0000","last_active":"2024-09-17T03:14:56.351063+0000","last_peered":"2024-09-17T03:14:56.351063+0000","last_clean":"2024-09-17T03:14:56.351063+0000","last_became_active":"2024-09-17T03:14:56.346207+0000","last_became_peered":"2024-09-17T03:14:56.346207+0000","last_unstale":"2024-09-17T03:14:56.351063+0000","last_undegraded":"2024-09-17T03:14:56.351063+0000","last_fullsized":"2024-09-17T03:14:56.351063+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:53.559531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351860+0000","last_change":"2024-09-17T03:14:56.341373+0000","last_active":"2024-09-17T03:14:56.351860+0000","last_peered":"2024-09-17T03:14:56.351860+0000","last_clean":"2024-09-17T03:14:56.351860+0000","last_became_active":"2024-09-17T03:14:56.341066+0000","last_became_peered":"2024-09-17T03:14:56.341066+0000","last_unstale":"2024-09-17T03:14:56.351860+0000","last_undegraded":"2024-09-17T03:14:56.351860+0000","last_fullsized":"2024-09-17T03:14:56.351860+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:28.743694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.354608+0000","last_change":"2024-09-17T03:14:55.321527+0000","last_active":"2024-09-17T03:14:56.354608+0000","last_peered":"2024-09-17T03:14:56.354608+0000","last_clean":"2024-09-17T03:14:56.354608+0000","last_became_active":"2024-09-17T03:14:55.320714+0000","last_became_peered":"2024-09-17T03:14:55.320714+0000","last_unstale":"2024-09-17T03:14:56.354608+0000","last_undegraded":"2024-09-17T03:14:56.354608+0000","last_fullsized":"2024-09-17T03:14:56.354608+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:16:27.821016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351470+0000","last_change":"2024-09-17T03:14:56.344463+0000","last_active":"2024-09-17T03:14:56.351470+0000","last_peered":"2024-09-17T03:14:56.351470+0000","last_clean":"2024-09-17T03:14:56.351470+0000","last_became_active":"2024-09-17T03:14:56.344125+0000","last_became_peered":"2024-09-17T03:14:56.344125+0000","last_unstale":"2024-09-17T03:14:56.351470+0000","last_undegraded":"2024-09-17T03:14:56.351470+0000","last_fullsized":"2024-09-17T03:14:56.351470+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:05.079987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T03:14:54.312526+0000","last_change":"2024-09-17T03:14:54.312526+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.312526+0000","last_undegraded":"2024-09-17T03:14:54.312526+0000","last_fullsized":"2024-09-17T03:14:54.312526+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.356182+0000","last_change":"2024-09-17T03:14:56.354666+0000","last_active":"2024-09-17T03:14:56.356182+0000","last_peered":"2024-09-17T03:14:56.356182+0000","last_clean":"2024-09-17T03:14:56.356182+0000","last_became_active":"2024-09-17T03:14:56.354360+0000","last_became_peered":"2024-09-17T03:14:56.354360+0000","last_unstale":"2024-09-17T03:14:56.356182+0000","last_undegraded":"2024-09-17T03:14:56.356182+0000","last_fullsized":"2024-09-17T03:14:56.356182+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:19.990834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341309+0000","last_change":"2024-09-17T03:14:55.332084+0000","last_active":"2024-09-17T03:14:57.341309+0000","last_peered":"2024-09-17T03:14:57.341309+0000","last_clean":"2024-09-17T03:14:57.341309+0000","last_became_active":"2024-09-17T03:14:55.331735+0000","last_became_peered":"2024-09-17T03:14:55.331735+0000","last_unstale":"2024-09-17T03:14:57.341309+0000","last_undegraded":"2024-09-17T03:14:57.341309+0000","last_fullsized":"2024-09-17T03:14:57.341309+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:23.002081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T03:14:54.312074+0000","last_change":"2024-09-17T03:14:54.312074+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.312074+0000","last_undegraded":"2024-09-17T03:14:54.312074+0000","last_fullsized":"2024-09-17T03:14:54.312074+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.348195+0000","last_change":"2024-09-17T03:14:56.345924+0000","last_active":"2024-09-17T03:14:56.348195+0000","last_peered":"2024-09-17T03:14:56.348195+0000","last_clean":"2024-09-17T03:14:56.348195+0000","last_became_active":"2024-09-17T03:14:56.345340+0000","last_became_peered":"2024-09-17T03:14:56.345340+0000","last_unstale":"2024-09-17T03:14:56.348195+0000","last_undegraded":"2024-09-17T03:14:56.348195+0000","last_fullsized":"2024-09-17T03:14:56.348195+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:18.485654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.357022+0000","last_change":"2024-09-17T03:14:55.334211+0000","last_active":"2024-09-17T03:14:56.357022+0000","last_peered":"2024-09-17T03:14:56.357022+0000","last_clean":"2024-09-17T03:14:56.357022+0000","last_became_active":"2024-09-17T03:14:55.333714+0000","last_became_peered":"2024-09-17T03:14:55.333714+0000","last_unstale":"2024-09-17T03:14:56.357022+0000","last_undegraded":"2024-09-17T03:14:56.357022+0000","last_fullsized":"2024-09-17T03:14:56.357022+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:30:39.180289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.353185+0000","last_change":"2024-09-17T03:14:56.348919+0000","last_active":"2024-09-17T03:14:56.353185+0000","last_peered":"2024-09-17T03:14:56.353185+0000","last_clean":"2024-09-17T03:14:56.353185+0000","last_became_active":"2024-09-17T03:14:56.348235+0000","last_became_peered":"2024-09-17T03:14:56.348235+0000","last_unstale":"2024-09-17T03:14:56.353185+0000","last_undegraded":"2024-09-17T03:14:56.353185+0000","last_fullsized":"2024-09-17T03:14:56.353185+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:14:16.498828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.357606+0000","last_change":"2024-09-17T03:14:55.335181+0000","last_active":"2024-09-17T03:14:56.357606+0000","last_peered":"2024-09-17T03:14:56.357606+0000","last_clean":"2024-09-17T03:14:56.357606+0000","last_became_active":"2024-09-17T03:14:55.334860+0000","last_became_peered":"2024-09-17T03:14:55.334860+0000","last_unstale":"2024-09-17T03:14:56.357606+0000","last_undegraded":"2024-09-17T03:14:56.357606+0000","last_fullsized":"2024-09-17T03:14:56.357606+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:34:41.387404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.341334+0000","last_change":"2024-09-17T03:14:56.338147+0000","last_active":"2024-09-17T03:14:56.341334+0000","last_peered":"2024-09-17T03:14:56.341334+0000","last_clean":"2024-09-17T03:14:56.341334+0000","last_became_active":"2024-09-17T03:14:56.337801+0000","last_became_peered":"2024-09-17T03:14:56.337801+0000","last_unstale":"2024-09-17T03:14:56.341334+0000","last_undegraded":"2024-09-17T03:14:56.341334+0000","last_fullsized":"2024-09-17T03:14:56.341334+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:24.007873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.340815+0000","last_change":"2024-09-17T03:14:55.328132+0000","last_active":"2024-09-17T03:14:57.340815+0000","last_peered":"2024-09-17T03:14:57.340815+0000","last_clean":"2024-09-17T03:14:57.340815+0000","last_became_active":"2024-09-17T03:14:55.327773+0000","last_became_peered":"2024-09-17T03:14:55.327773+0000","last_unstale":"2024-09-17T03:14:57.340815+0000","last_undegraded":"2024-09-17T03:14:57.340815+0000","last_fullsized":"2024-09-17T03:14:57.340815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:09.899391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.330123+0000","last_change":"2024-09-17T03:14:55.334322+0000","last_active":"2024-09-17T03:14:56.330123+0000","last_peered":"2024-09-17T03:14:56.330123+0000","last_clean":"2024-09-17T03:14:56.330123+0000","last_became_active":"2024-09-17T03:14:55.333947+0000","last_became_peered":"2024-09-17T03:14:55.333947+0000","last_unstale":"2024-09-17T03:14:56.330123+0000","last_undegraded":"2024-09-17T03:14:56.330123+0000","last_fullsized":"2024-09-17T03:14:56.330123+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:44.089548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351133+0000","last_change":"2024-09-17T03:14:56.345419+0000","last_active":"2024-09-17T03:14:56.351133+0000","last_peered":"2024-09-17T03:14:56.351133+0000","last_clean":"2024-09-17T03:14:56.351133+0000","last_became_active":"2024-09-17T03:14:56.343640+0000","last_became_peered":"2024-09-17T03:14:56.343640+0000","last_unstale":"2024-09-17T03:14:56.351133+0000","last_undegraded":"2024-09-17T03:14:56.351133+0000","last_fullsized":"2024-09-17T03:14:56.351133+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:20.330393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344278+0000","last_change":"2024-09-17T03:14:55.323288+0000","last_active":"2024-09-17T03:14:57.344278+0000","last_peered":"2024-09-17T03:14:57.344278+0000","last_clean":"2024-09-17T03:14:57.344278+0000","last_became_active":"2024-09-17T03:14:55.322723+0000","last_became_peered":"2024-09-17T03:14:55.322723+0000","last_unstale":"2024-09-17T03:14:57.344278+0000","last_undegraded":"2024-09-17T03:14:57.344278+0000","last_fullsized":"2024-09-17T03:14:57.344278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:11:24.012880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344030+0000","last_change":"2024-09-17T03:14:55.325887+0000","last_active":"2024-09-17T03:14:57.344030+0000","last_peered":"2024-09-17T03:14:57.344030+0000","last_clean":"2024-09-17T03:14:57.344030+0000","last_became_active":"2024-09-17T03:14:55.325563+0000","last_became_peered":"2024-09-17T03:14:55.325563+0000","last_unstale":"2024-09-17T03:14:57.344030+0000","last_undegraded":"2024-09-17T03:14:57.344030+0000","last_fullsized":"2024-09-17T03:14:57.344030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:22.517631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.354205+0000","last_change":"2024-09-17T03:14:55.329954+0000","last_active":"2024-09-17T03:14:56.354205+0000","last_peered":"2024-09-17T03:14:56.354205+0000","last_clean":"2024-09-17T03:14:56.354205+0000","last_became_active":"2024-09-17T03:14:55.329631+0000","last_became_peered":"2024-09-17T03:14:55.329631+0000","last_unstale":"2024-09-17T03:14:56.354205+0000","last_undegraded":"2024-09-17T03:14:56.354205+0000","last_fullsized":"2024-09-17T03:14:56.354205+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:45.028465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.354624+0000","last_change":"2024-09-17T03:14:56.352353+0000","last_active":"2024-09-17T03:14:56.354624+0000","last_peered":"2024-09-17T03:14:56.354624+0000","last_clean":"2024-09-17T03:14:56.354624+0000","last_became_active":"2024-09-17T03:14:56.352041+0000","last_became_peered":"2024-09-17T03:14:56.352041+0000","last_unstale":"2024-09-17T03:14:56.354624+0000","last_undegraded":"2024-09-17T03:14:56.354624+0000","last_fullsized":"2024-09-17T03:14:56.354624+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:43:07.926923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.360502+0000","last_change":"2024-09-17T03:14:55.332350+0000","last_active":"2024-09-17T03:14:56.360502+0000","last_peered":"2024-09-17T03:14:56.360502+0000","last_clean":"2024-09-17T03:14:56.360502+0000","last_became_active":"2024-09-17T03:14:55.332010+0000","last_became_peered":"2024-09-17T03:14:55.332010+0000","last_unstale":"2024-09-17T03:14:56.360502+0000","last_undegraded":"2024-09-17T03:14:56.360502+0000","last_fullsized":"2024-09-17T03:14:56.360502+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:19.084170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351935+0000","last_change":"2024-09-17T03:14:56.344117+0000","last_active":"2024-09-17T03:14:56.351935+0000","last_peered":"2024-09-17T03:14:56.351935+0000","last_clean":"2024-09-17T03:14:56.351935+0000","last_became_active":"2024-09-17T03:14:56.343821+0000","last_became_peered":"2024-09-17T03:14:56.343821+0000","last_unstale":"2024-09-17T03:14:56.351935+0000","last_undegraded":"2024-09-17T03:14:56.351935+0000","last_fullsized":"2024-09-17T03:14:56.351935+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:25.974302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.355844+0000","last_change":"2024-09-17T03:14:55.333495+0000","last_active":"2024-09-17T03:14:56.355844+0000","last_peered":"2024-09-17T03:14:56.355844+0000","last_clean":"2024-09-17T03:14:56.355844+0000","last_became_active":"2024-09-17T03:14:55.332779+0000","last_became_peered":"2024-09-17T03:14:55.332779+0000","last_unstale":"2024-09-17T03:14:56.355844+0000","last_undegraded":"2024-09-17T03:14:56.355844+0000","last_fullsized":"2024-09-17T03:14:56.355844+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:19.084170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T03:14:54.311860+0000","last_change":"2024-09-17T03:14:54.311860+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.311860+0000","last_undegraded":"2024-09-17T03:14:54.311860+0000","last_fullsized":"2024-09-17T03:14:54.311860+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.345568+0000","last_change":"2024-09-17T03:14:56.341404+0000","last_active":"2024-09-17T03:14:56.345568+0000","last_peered":"2024-09-17T03:14:56.345568+0000","last_clean":"2024-09-17T03:14:56.345568+0000","last_became_active":"2024-09-17T03:14:56.341086+0000","last_became_peered":"2024-09-17T03:14:56.341086+0000","last_unstale":"2024-09-17T03:14:56.345568+0000","last_undegraded":"2024-09-17T03:14:56.345568+0000","last_fullsized":"2024-09-17T03:14:56.345568+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:19:29.104716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T03:14:54.313012+0000","last_change":"2024-09-17T03:14:54.313012+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.313012+0000","last_undegraded":"2024-09-17T03:14:54.313012+0000","last_fullsized":"2024-09-17T03:14:54.313012+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351204+0000","last_change":"2024-09-17T03:14:56.347429+0000","last_active":"2024-09-17T03:14:56.351204+0000","last_peered":"2024-09-17T03:14:56.351204+0000","last_clean":"2024-09-17T03:14:56.351204+0000","last_became_active":"2024-09-17T03:14:56.347101+0000","last_became_peered":"2024-09-17T03:14:56.347101+0000","last_unstale":"2024-09-17T03:14:56.351204+0000","last_undegraded":"2024-09-17T03:14:56.351204+0000","last_fullsized":"2024-09-17T03:14:56.351204+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:46.793182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.340674+0000","last_change":"2024-09-17T03:14:55.330923+0000","last_active":"2024-09-17T03:14:57.340674+0000","last_peered":"2024-09-17T03:14:57.340674+0000","last_clean":"2024-09-17T03:14:57.340674+0000","last_became_active":"2024-09-17T03:14:55.330601+0000","last_became_peered":"2024-09-17T03:14:55.330601+0000","last_unstale":"2024-09-17T03:14:57.340674+0000","last_undegraded":"2024-09-17T03:14:57.340674+0000","last_fullsized":"2024-09-17T03:14:57.340674+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:21.326711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351107+0000","last_change":"2024-09-17T03:14:56.343137+0000","last_active":"2024-09-17T03:14:56.351107+0000","last_peered":"2024-09-17T03:14:56.351107+0000","last_clean":"2024-09-17T03:14:56.351107+0000","last_became_active":"2024-09-17T03:14:56.342838+0000","last_became_peered":"2024-09-17T03:14:56.342838+0000","last_unstale":"2024-09-17T03:14:56.351107+0000","last_undegraded":"2024-09-17T03:14:56.351107+0000","last_fullsized":"2024-09-17T03:14:56.351107+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:54:34.725205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341150+0000","last_change":"2024-09-17T03:14:55.331984+0000","last_active":"2024-09-17T03:14:57.341150+0000","last_peered":"2024-09-17T03:14:57.341150+0000","last_clean":"2024-09-17T03:14:57.341150+0000","last_became_active":"2024-09-17T03:14:55.331543+0000","last_became_peered":"2024-09-17T03:14:55.331543+0000","last_unstale":"2024-09-17T03:14:57.341150+0000","last_undegraded":"2024-09-17T03:14:57.341150+0000","last_fullsized":"2024-09-17T03:14:57.341150+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:06:37.330728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.338642+0000","last_change":"2024-09-17T03:14:56.336542+0000","last_active":"2024-09-17T03:14:56.338642+0000","last_peered":"2024-09-17T03:14:56.338642+0000","last_clean":"2024-09-17T03:14:56.338642+0000","last_became_active":"2024-09-17T03:14:56.336216+0000","last_became_peered":"2024-09-17T03:14:56.336216+0000","last_unstale":"2024-09-17T03:14:56.338642+0000","last_undegraded":"2024-09-17T03:14:56.338642+0000","last_fullsized":"2024-09-17T03:14:56.338642+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:10.905183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.355337+0000","last_change":"2024-09-17T03:14:55.335726+0000","last_active":"2024-09-17T03:14:56.355337+0000","last_peered":"2024-09-17T03:14:56.355337+0000","last_clean":"2024-09-17T03:14:56.355337+0000","last_became_active":"2024-09-17T03:14:55.335268+0000","last_became_peered":"2024-09-17T03:14:55.335268+0000","last_unstale":"2024-09-17T03:14:56.355337+0000","last_undegraded":"2024-09-17T03:14:56.355337+0000","last_fullsized":"2024-09-17T03:14:56.355337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:13.660225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344275+0000","last_change":"2024-09-17T03:14:56.336628+0000","last_active":"2024-09-17T03:14:57.344275+0000","last_peered":"2024-09-17T03:14:57.344275+0000","last_clean":"2024-09-17T03:14:57.344275+0000","last_became_active":"2024-09-17T03:14:56.336330+0000","last_became_peered":"2024-09-17T03:14:56.336330+0000","last_unstale":"2024-09-17T03:14:57.344275+0000","last_undegraded":"2024-09-17T03:14:57.344275+0000","last_fullsized":"2024-09-17T03:14:57.344275+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:02:13.758412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351038+0000","last_change":"2024-09-17T03:14:55.324166+0000","last_active":"2024-09-17T03:14:56.351038+0000","last_peered":"2024-09-17T03:14:56.351038+0000","last_clean":"2024-09-17T03:14:56.351038+0000","last_became_active":"2024-09-17T03:14:55.323844+0000","last_became_peered":"2024-09-17T03:14:55.323844+0000","last_unstale":"2024-09-17T03:14:56.351038+0000","last_undegraded":"2024-09-17T03:14:56.351038+0000","last_fullsized":"2024-09-17T03:14:56.351038+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:27:53.850399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341186+0000","last_change":"2024-09-17T03:14:56.337913+0000","last_active":"2024-09-17T03:14:57.341186+0000","last_peered":"2024-09-17T03:14:57.341186+0000","last_clean":"2024-09-17T03:14:57.341186+0000","last_became_active":"2024-09-17T03:14:56.337610+0000","last_became_peered":"2024-09-17T03:14:56.337610+0000","last_unstale":"2024-09-17T03:14:57.341186+0000","last_undegraded":"2024-09-17T03:14:57.341186+0000","last_fullsized":"2024-09-17T03:14:57.341186+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:08.606153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.325094+0000","last_change":"2024-09-17T03:14:55.333099+0000","last_active":"2024-09-17T03:14:56.325094+0000","last_peered":"2024-09-17T03:14:56.325094+0000","last_clean":"2024-09-17T03:14:56.325094+0000","last_became_active":"2024-09-17T03:14:55.332738+0000","last_became_peered":"2024-09-17T03:14:55.332738+0000","last_unstale":"2024-09-17T03:14:56.325094+0000","last_undegraded":"2024-09-17T03:14:56.325094+0000","last_fullsized":"2024-09-17T03:14:56.325094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:50.915622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.344808+0000","last_change":"2024-09-17T03:14:56.340119+0000","last_active":"2024-09-17T03:14:56.344808+0000","last_peered":"2024-09-17T03:14:56.344808+0000","last_clean":"2024-09-17T03:14:56.344808+0000","last_became_active":"2024-09-17T03:14:56.339795+0000","last_became_peered":"2024-09-17T03:14:56.339795+0000","last_unstale":"2024-09-17T03:14:56.344808+0000","last_undegraded":"2024-09-17T03:14:56.344808+0000","last_fullsized":"2024-09-17T03:14:56.344808+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:04.002930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344218+0000","last_change":"2024-09-17T03:14:55.327065+0000","last_active":"2024-09-17T03:14:57.344218+0000","last_peered":"2024-09-17T03:14:57.344218+0000","last_clean":"2024-09-17T03:14:57.344218+0000","last_became_active":"2024-09-17T03:14:55.326774+0000","last_became_peered":"2024-09-17T03:14:55.326774+0000","last_unstale":"2024-09-17T03:14:57.344218+0000","last_undegraded":"2024-09-17T03:14:57.344218+0000","last_fullsized":"2024-09-17T03:14:57.344218+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:37.138244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344009+0000","last_change":"2024-09-17T03:14:56.335950+0000","last_active":"2024-09-17T03:14:57.344009+0000","last_peered":"2024-09-17T03:14:57.344009+0000","last_clean":"2024-09-17T03:14:57.344009+0000","last_became_active":"2024-09-17T03:14:56.335641+0000","last_became_peered":"2024-09-17T03:14:56.335641+0000","last_unstale":"2024-09-17T03:14:57.344009+0000","last_undegraded":"2024-09-17T03:14:57.344009+0000","last_fullsized":"2024-09-17T03:14:57.344009+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:03.609971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.329227+0000","last_change":"2024-09-17T03:14:55.335625+0000","last_active":"2024-09-17T03:14:56.329227+0000","last_peered":"2024-09-17T03:14:56.329227+0000","last_clean":"2024-09-17T03:14:56.329227+0000","last_became_active":"2024-09-17T03:14:55.335317+0000","last_became_peered":"2024-09-17T03:14:55.335317+0000","last_unstale":"2024-09-17T03:14:56.329227+0000","last_undegraded":"2024-09-17T03:14:56.329227+0000","last_fullsized":"2024-09-17T03:14:56.329227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:47.254999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.357401+0000","last_change":"2024-09-17T03:14:56.346898+0000","last_active":"2024-09-17T03:14:56.357401+0000","last_peered":"2024-09-17T03:14:56.357401+0000","last_clean":"2024-09-17T03:14:56.357401+0000","last_became_active":"2024-09-17T03:14:56.346603+0000","last_became_peered":"2024-09-17T03:14:56.346603+0000","last_unstale":"2024-09-17T03:14:56.357401+0000","last_undegraded":"2024-09-17T03:14:56.357401+0000","last_fullsized":"2024-09-17T03:14:56.357401+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:26:13.257738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.328281+0000","last_change":"2024-09-17T03:14:55.328041+0000","last_active":"2024-09-17T03:14:56.328281+0000","last_peered":"2024-09-17T03:14:56.328281+0000","last_clean":"2024-09-17T03:14:56.328281+0000","last_became_active":"2024-09-17T03:14:55.327533+0000","last_became_peered":"2024-09-17T03:14:55.327533+0000","last_unstale":"2024-09-17T03:14:56.328281+0000","last_undegraded":"2024-09-17T03:14:56.328281+0000","last_fullsized":"2024-09-17T03:14:56.328281+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:21.326711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.337735+0000","last_change":"2024-09-17T03:14:56.336599+0000","last_active":"2024-09-17T03:14:56.337735+0000","last_peered":"2024-09-17T03:14:56.337735+0000","last_clean":"2024-09-17T03:14:56.337735+0000","last_became_active":"2024-09-17T03:14:56.336243+0000","last_became_peered":"2024-09-17T03:14:56.336243+0000","last_unstale":"2024-09-17T03:14:56.337735+0000","last_undegraded":"2024-09-17T03:14:56.337735+0000","last_fullsized":"2024-09-17T03:14:56.337735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:06:38.336520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.329616+0000","last_change":"2024-09-17T03:14:55.324984+0000","last_active":"2024-09-17T03:14:56.329616+0000","last_peered":"2024-09-17T03:14:56.329616+0000","last_clean":"2024-09-17T03:14:56.329616+0000","last_became_active":"2024-09-17T03:14:55.324696+0000","last_became_peered":"2024-09-17T03:14:55.324696+0000","last_unstale":"2024-09-17T03:14:56.329616+0000","last_undegraded":"2024-09-17T03:14:56.329616+0000","last_fullsized":"2024-09-17T03:14:56.329616+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:10.371792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341211+0000","last_change":"2024-09-17T03:14:56.338902+0000","last_active":"2024-09-17T03:14:57.341211+0000","last_peered":"2024-09-17T03:14:57.341211+0000","last_clean":"2024-09-17T03:14:57.341211+0000","last_became_active":"2024-09-17T03:14:56.338504+0000","last_became_peered":"2024-09-17T03:14:56.338504+0000","last_unstale":"2024-09-17T03:14:57.341211+0000","last_undegraded":"2024-09-17T03:14:57.341211+0000","last_fullsized":"2024-09-17T03:14:57.341211+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:16.330515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.357201+0000","last_change":"2024-09-17T03:14:55.328618+0000","last_active":"2024-09-17T03:14:56.357201+0000","last_peered":"2024-09-17T03:14:56.357201+0000","last_clean":"2024-09-17T03:14:56.357201+0000","last_became_active":"2024-09-17T03:14:55.328321+0000","last_became_peered":"2024-09-17T03:14:55.328321+0000","last_unstale":"2024-09-17T03:14:56.357201+0000","last_undegraded":"2024-09-17T03:14:56.357201+0000","last_fullsized":"2024-09-17T03:14:56.357201+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:30:39.180289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.354677+0000","last_change":"2024-09-17T03:14:56.341818+0000","last_active":"2024-09-17T03:14:56.354677+0000","last_peered":"2024-09-17T03:14:56.354677+0000","last_clean":"2024-09-17T03:14:56.354677+0000","last_became_active":"2024-09-17T03:14:56.341526+0000","last_became_peered":"2024-09-17T03:14:56.341526+0000","last_unstale":"2024-09-17T03:14:56.354677+0000","last_undegraded":"2024-09-17T03:14:56.354677+0000","last_fullsized":"2024-09-17T03:14:56.354677+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:35.033454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344149+0000","last_change":"2024-09-17T03:14:55.325076+0000","last_active":"2024-09-17T03:14:57.344149+0000","last_peered":"2024-09-17T03:14:57.344149+0000","last_clean":"2024-09-17T03:14:57.344149+0000","last_became_active":"2024-09-17T03:14:55.324738+0000","last_became_peered":"2024-09-17T03:14:55.324738+0000","last_unstale":"2024-09-17T03:14:57.344149+0000","last_undegraded":"2024-09-17T03:14:57.344149+0000","last_fullsized":"2024-09-17T03:14:57.344149+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:26:33.226093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341372+0000","last_change":"2024-09-17T03:14:56.348451+0000","last_active":"2024-09-17T03:14:57.341372+0000","last_peered":"2024-09-17T03:14:57.341372+0000","last_clean":"2024-09-17T03:14:57.341372+0000","last_became_active":"2024-09-17T03:14:56.348169+0000","last_became_peered":"2024-09-17T03:14:56.348169+0000","last_unstale":"2024-09-17T03:14:57.341372+0000","last_undegraded":"2024-09-17T03:14:57.341372+0000","last_fullsized":"2024-09-17T03:14:57.341372+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:47.114511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.332977+0000","last_change":"2024-09-17T03:14:55.327511+0000","last_active":"2024-09-17T03:14:56.332977+0000","last_peered":"2024-09-17T03:14:56.332977+0000","last_clean":"2024-09-17T03:14:56.332977+0000","last_became_active":"2024-09-17T03:14:55.327195+0000","last_became_peered":"2024-09-17T03:14:55.327195+0000","last_unstale":"2024-09-17T03:14:56.332977+0000","last_undegraded":"2024-09-17T03:14:56.332977+0000","last_fullsized":"2024-09-17T03:14:56.332977+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:13.660225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.344622+0000","last_change":"2024-09-17T03:14:56.337722+0000","last_active":"2024-09-17T03:14:56.344622+0000","last_peered":"2024-09-17T03:14:56.344622+0000","last_clean":"2024-09-17T03:14:56.344622+0000","last_became_active":"2024-09-17T03:14:56.337353+0000","last_became_peered":"2024-09-17T03:14:56.337353+0000","last_unstale":"2024-09-17T03:14:56.344622+0000","last_undegraded":"2024-09-17T03:14:56.344622+0000","last_fullsized":"2024-09-17T03:14:56.344622+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:34:03.584090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.337538+0000","last_change":"2024-09-17T03:14:55.328438+0000","last_active":"2024-09-17T03:14:56.337538+0000","last_peered":"2024-09-17T03:14:56.337538+0000","last_clean":"2024-09-17T03:14:56.337538+0000","last_became_active":"2024-09-17T03:14:55.328158+0000","last_became_peered":"2024-09-17T03:14:55.328158+0000","last_unstale":"2024-09-17T03:14:56.337538+0000","last_undegraded":"2024-09-17T03:14:56.337538+0000","last_fullsized":"2024-09-17T03:14:56.337538+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:19:28.098924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.331672+0000","last_change":"2024-09-17T03:14:55.330495+0000","last_active":"2024-09-17T03:14:56.331672+0000","last_peered":"2024-09-17T03:14:56.331672+0000","last_clean":"2024-09-17T03:14:56.331672+0000","last_became_active":"2024-09-17T03:14:55.330159+0000","last_became_peered":"2024-09-17T03:14:55.330159+0000","last_unstale":"2024-09-17T03:14:56.331672+0000","last_undegraded":"2024-09-17T03:14:56.331672+0000","last_fullsized":"2024-09-17T03:14:56.331672+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:55.212014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.331656+0000","last_change":"2024-09-17T03:14:55.331670+0000","last_active":"2024-09-17T03:14:56.331656+0000","last_peered":"2024-09-17T03:14:56.331656+0000","last_clean":"2024-09-17T03:14:56.331656+0000","last_became_active":"2024-09-17T03:14:55.331359+0000","last_became_peered":"2024-09-17T03:14:55.331359+0000","last_unstale":"2024-09-17T03:14:56.331656+0000","last_undegraded":"2024-09-17T03:14:56.331656+0000","last_fullsized":"2024-09-17T03:14:56.331656+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:55.212014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351181+0000","last_change":"2024-09-17T03:14:56.341402+0000","last_active":"2024-09-17T03:14:56.351181+0000","last_peered":"2024-09-17T03:14:56.351181+0000","last_clean":"2024-09-17T03:14:56.351181+0000","last_became_active":"2024-09-17T03:14:56.341095+0000","last_became_peered":"2024-09-17T03:14:56.341095+0000","last_unstale":"2024-09-17T03:14:56.351181+0000","last_undegraded":"2024-09-17T03:14:56.351181+0000","last_fullsized":"2024-09-17T03:14:56.351181+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:33:09.441087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T03:14:54.312546+0000","last_change":"2024-09-17T03:14:54.312546+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.312546+0000","last_undegraded":"2024-09-17T03:14:54.312546+0000","last_fullsized":"2024-09-17T03:14:54.312546+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.343160+0000","last_change":"2024-09-17T03:14:56.337604+0000","last_active":"2024-09-17T03:14:56.343160+0000","last_peered":"2024-09-17T03:14:56.343160+0000","last_clean":"2024-09-17T03:14:56.343160+0000","last_became_active":"2024-09-17T03:14:56.337283+0000","last_became_peered":"2024-09-17T03:14:56.337283+0000","last_unstale":"2024-09-17T03:14:56.343160+0000","last_undegraded":"2024-09-17T03:14:56.343160+0000","last_fullsized":"2024-09-17T03:14:56.343160+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:17.535415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.333475+0000","last_change":"2024-09-17T03:14:55.322959+0000","last_active":"2024-09-17T03:14:56.333475+0000","last_peered":"2024-09-17T03:14:56.333475+0000","last_clean":"2024-09-17T03:14:56.333475+0000","last_became_active":"2024-09-17T03:14:55.322475+0000","last_became_peered":"2024-09-17T03:14:55.322475+0000","last_unstale":"2024-09-17T03:14:56.333475+0000","last_undegraded":"2024-09-17T03:14:56.333475+0000","last_fullsized":"2024-09-17T03:14:56.333475+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:57.870235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":127,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.328846+0000","last_change":"2024-09-17T03:14:04.862167+0000","last_active":"2024-09-17T03:14:56.328846+0000","last_peered":"2024-09-17T03:14:56.328846+0000","last_clean":"2024-09-17T03:14:56.328846+0000","last_became_active":"2024-09-17T03:14:04.861731+0000","last_became_peered":"2024-09-17T03:14:04.861731+0000","last_unstale":"2024-09-17T03:14:56.328846+0000","last_undegraded":"2024-09-17T03:14:56.328846+0000","last_fullsized":"2024-09-17T03:14:56.328846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_clean_scrub_stamp":"2024-09-17T03:14:03.847721+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:32.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":104,"acting":104,"num_store_stats":7},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509456,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":368640,"data_stored":139029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":376832,"data_stored":140619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509455,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":962560,"data_stored":729957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":380928,"data_stored":139783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":368640,"data_stored":139041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":962560,"data_stored":729445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":368640,"data_stored":139097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":327680,"data_stored":120140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:15:00.040 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3c is not active+clean 2024-09-17T03:15:00.040 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:55.307935+0000', 'last_change': '2024-09-17T03:14:55.307935+0000', 'last_active': '2024-09-17T03:14:55.307935+0000', 'last_peered': '2024-09-17T03:14:55.307935+0000', 'last_clean': '2024-09-17T03:14:55.307935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:55.307935+0000', 'last_undegraded': '2024-09-17T03:14:55.307935+0000', 'last_fullsized': '2024-09-17T03:14:55.307935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:00.040 INFO:tasks.ceph.ceph_manager.ceph:PG 5.36 is not active+clean 2024-09-17T03:15:00.040 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:55.307935+0000', 'last_change': '2024-09-17T03:14:55.307935+0000', 'last_active': '2024-09-17T03:14:55.307935+0000', 'last_peered': '2024-09-17T03:14:55.307935+0000', 'last_clean': '2024-09-17T03:14:55.307935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:55.307935+0000', 'last_undegraded': '2024-09-17T03:14:55.307935+0000', 'last_fullsized': '2024-09-17T03:14:55.307935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:00.040 INFO:tasks.ceph.ceph_manager.ceph:PG 5.32 is not active+clean 2024-09-17T03:15:00.041 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:55.307935+0000', 'last_change': '2024-09-17T03:14:55.307935+0000', 'last_active': '2024-09-17T03:14:55.307935+0000', 'last_peered': '2024-09-17T03:14:55.307935+0000', 'last_clean': '2024-09-17T03:14:55.307935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:55.307935+0000', 'last_undegraded': '2024-09-17T03:14:55.307935+0000', 'last_fullsized': '2024-09-17T03:14:55.307935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:00.041 INFO:tasks.ceph.ceph_manager.ceph:PG 5.27 is not active+clean 2024-09-17T03:15:00.041 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:55.307935+0000', 'last_change': '2024-09-17T03:14:55.307935+0000', 'last_active': '2024-09-17T03:14:55.307935+0000', 'last_peered': '2024-09-17T03:14:55.307935+0000', 'last_clean': '2024-09-17T03:14:55.307935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:55.307935+0000', 'last_undegraded': '2024-09-17T03:14:55.307935+0000', 'last_fullsized': '2024-09-17T03:14:55.307935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:00.041 INFO:tasks.ceph.ceph_manager.ceph:PG 5.26 is not active+clean 2024-09-17T03:15:00.041 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:55.307935+0000', 'last_change': '2024-09-17T03:14:55.307935+0000', 'last_active': '2024-09-17T03:14:55.307935+0000', 'last_peered': '2024-09-17T03:14:55.307935+0000', 'last_clean': '2024-09-17T03:14:55.307935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:55.307935+0000', 'last_undegraded': '2024-09-17T03:14:55.307935+0000', 'last_fullsized': '2024-09-17T03:14:55.307935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:00.041 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1e is not active+clean 2024-09-17T03:15:00.041 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:54.312526+0000', 'last_change': '2024-09-17T03:14:54.312526+0000', 'last_active': '2024-09-17T03:14:54.302142+0000', 'last_peered': '2024-09-17T03:14:54.302142+0000', 'last_clean': '2024-09-17T03:14:54.302142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:54.312526+0000', 'last_undegraded': '2024-09-17T03:14:54.312526+0000', 'last_fullsized': '2024-09-17T03:14:54.312526+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:54.302142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:54.302142+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:54.302142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T03:15:00.041 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1d is not active+clean 2024-09-17T03:15:00.041 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:55.307935+0000', 'last_change': '2024-09-17T03:14:55.307935+0000', 'last_active': '2024-09-17T03:14:55.307935+0000', 'last_peered': '2024-09-17T03:14:55.307935+0000', 'last_clean': '2024-09-17T03:14:55.307935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:55.307935+0000', 'last_undegraded': '2024-09-17T03:14:55.307935+0000', 'last_fullsized': '2024-09-17T03:14:55.307935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:00.041 INFO:tasks.ceph.ceph_manager.ceph:PG 4.1c is not active+clean 2024-09-17T03:15:00.041 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:54.312074+0000', 'last_change': '2024-09-17T03:14:54.312074+0000', 'last_active': '2024-09-17T03:14:54.302142+0000', 'last_peered': '2024-09-17T03:14:54.302142+0000', 'last_clean': '2024-09-17T03:14:54.302142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:54.312074+0000', 'last_undegraded': '2024-09-17T03:14:54.312074+0000', 'last_fullsized': '2024-09-17T03:14:54.312074+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:54.302142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:54.302142+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:54.302142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T03:15:00.042 INFO:tasks.ceph.ceph_manager.ceph:PG 5.19 is not active+clean 2024-09-17T03:15:00.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:55.307935+0000', 'last_change': '2024-09-17T03:14:55.307935+0000', 'last_active': '2024-09-17T03:14:55.307935+0000', 'last_peered': '2024-09-17T03:14:55.307935+0000', 'last_clean': '2024-09-17T03:14:55.307935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:55.307935+0000', 'last_undegraded': '2024-09-17T03:14:55.307935+0000', 'last_fullsized': '2024-09-17T03:14:55.307935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:00.042 INFO:tasks.ceph.ceph_manager.ceph:PG 5.17 is not active+clean 2024-09-17T03:15:00.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:55.307935+0000', 'last_change': '2024-09-17T03:14:55.307935+0000', 'last_active': '2024-09-17T03:14:55.307935+0000', 'last_peered': '2024-09-17T03:14:55.307935+0000', 'last_clean': '2024-09-17T03:14:55.307935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:55.307935+0000', 'last_undegraded': '2024-09-17T03:14:55.307935+0000', 'last_fullsized': '2024-09-17T03:14:55.307935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:00.042 INFO:tasks.ceph.ceph_manager.ceph:PG 5.14 is not active+clean 2024-09-17T03:15:00.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:55.307935+0000', 'last_change': '2024-09-17T03:14:55.307935+0000', 'last_active': '2024-09-17T03:14:55.307935+0000', 'last_peered': '2024-09-17T03:14:55.307935+0000', 'last_clean': '2024-09-17T03:14:55.307935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:55.307935+0000', 'last_undegraded': '2024-09-17T03:14:55.307935+0000', 'last_fullsized': '2024-09-17T03:14:55.307935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:00.042 INFO:tasks.ceph.ceph_manager.ceph:PG 5.13 is not active+clean 2024-09-17T03:15:00.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:55.307935+0000', 'last_change': '2024-09-17T03:14:55.307935+0000', 'last_active': '2024-09-17T03:14:55.307935+0000', 'last_peered': '2024-09-17T03:14:55.307935+0000', 'last_clean': '2024-09-17T03:14:55.307935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:55.307935+0000', 'last_undegraded': '2024-09-17T03:14:55.307935+0000', 'last_fullsized': '2024-09-17T03:14:55.307935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:00.042 INFO:tasks.ceph.ceph_manager.ceph:PG 4.12 is not active+clean 2024-09-17T03:15:00.042 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:54.311860+0000', 'last_change': '2024-09-17T03:14:54.311860+0000', 'last_active': '2024-09-17T03:14:54.302142+0000', 'last_peered': '2024-09-17T03:14:54.302142+0000', 'last_clean': '2024-09-17T03:14:54.302142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:54.311860+0000', 'last_undegraded': '2024-09-17T03:14:54.311860+0000', 'last_fullsized': '2024-09-17T03:14:54.311860+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:54.302142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:54.302142+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:54.302142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T03:15:00.043 INFO:tasks.ceph.ceph_manager.ceph:PG 4.11 is not active+clean 2024-09-17T03:15:00.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:54.313012+0000', 'last_change': '2024-09-17T03:14:54.313012+0000', 'last_active': '2024-09-17T03:14:54.302142+0000', 'last_peered': '2024-09-17T03:14:54.302142+0000', 'last_clean': '2024-09-17T03:14:54.302142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:54.313012+0000', 'last_undegraded': '2024-09-17T03:14:54.313012+0000', 'last_fullsized': '2024-09-17T03:14:54.313012+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:54.302142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:54.302142+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:54.302142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T03:15:00.043 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3 is not active+clean 2024-09-17T03:15:00.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:55.307935+0000', 'last_change': '2024-09-17T03:14:55.307935+0000', 'last_active': '2024-09-17T03:14:55.307935+0000', 'last_peered': '2024-09-17T03:14:55.307935+0000', 'last_clean': '2024-09-17T03:14:55.307935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:55.307935+0000', 'last_undegraded': '2024-09-17T03:14:55.307935+0000', 'last_fullsized': '2024-09-17T03:14:55.307935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:00.043 INFO:tasks.ceph.ceph_manager.ceph:PG 5.0 is not active+clean 2024-09-17T03:15:00.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:14:55.307935+0000', 'last_change': '2024-09-17T03:14:55.307935+0000', 'last_active': '2024-09-17T03:14:55.307935+0000', 'last_peered': '2024-09-17T03:14:55.307935+0000', 'last_clean': '2024-09-17T03:14:55.307935+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:55.307935+0000', 'last_undegraded': '2024-09-17T03:14:55.307935+0000', 'last_fullsized': '2024-09-17T03:14:55.307935+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:55.307935+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:00.043 INFO:tasks.ceph.ceph_manager.ceph:PG 4.0 is not active+clean 2024-09-17T03:15:00.043 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '4.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:14:54.312546+0000', 'last_change': '2024-09-17T03:14:54.312546+0000', 'last_active': '2024-09-17T03:14:54.302142+0000', 'last_peered': '2024-09-17T03:14:54.302142+0000', 'last_clean': '2024-09-17T03:14:54.302142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:14:54.312546+0000', 'last_undegraded': '2024-09-17T03:14:54.312546+0000', 'last_fullsized': '2024-09-17T03:14:54.312546+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:14:54.302142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:14:54.302142+0000', 'last_clean_scrub_stamp': '2024-09-17T03:14:54.302142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-17T03:15:00.044 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T03:15:00.143 INFO:tasks.ceph.mds.c.smithi087.stderr:2024-09-17T03:15:00.144+0000 7f18d0326640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T03:15:00.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.189+0000 7f9db0bc6640 1 -- 172.21.15.87:0/2719820169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dac160250 msgr2=0x7f9dac160660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:00.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.189+0000 7f9db0bc6640 1 --2- 172.21.15.87:0/2719820169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dac160250 0x7f9dac160660 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9dac074760 tx=0x7f9da402a980 comp rx=0 tx=0).stop 2024-09-17T03:15:00.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.189+0000 7f9db0bc6640 1 -- 172.21.15.87:0/2719820169 shutdown_connections 2024-09-17T03:15:00.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.189+0000 7f9db0bc6640 1 --2- 172.21.15.87:0/2719820169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dac160250 0x7f9dac160660 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:00.188 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.189+0000 7f9db0bc6640 1 --2- 172.21.15.87:0/2719820169 >> v2:172.21.15.149:3300/0 conn(0x7f9dac09aa30 0x7f9dac09ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:00.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.189+0000 7f9db0bc6640 1 --2- 172.21.15.87:0/2719820169 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9dac0ce0a0 0x7f9dac0d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:00.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.189+0000 7f9db0bc6640 1 -- 172.21.15.87:0/2719820169 >> 172.21.15.87:0/2719820169 conn(0x7f9dac0b3d30 msgr2=0x7f9dac0b4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:00.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.189+0000 7f9db0bc6640 1 -- 172.21.15.87:0/2719820169 shutdown_connections 2024-09-17T03:15:00.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.189+0000 7f9db0bc6640 1 -- 172.21.15.87:0/2719820169 wait complete. 2024-09-17T03:15:00.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.191+0000 7f9db0bc6640 1 Processor -- start 2024-09-17T03:15:00.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.191+0000 7f9db0bc6640 1 -- start start 2024-09-17T03:15:00.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.192+0000 7f9db0bc6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dac09aa30 0x7f9dac06f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:00.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.192+0000 7f9db0bc6640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9dac0ce0a0 0x7f9dac0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:00.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.192+0000 7f9db0bc6640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9dac160250 0x7f9dac15c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:00.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.192+0000 7f9db0bc6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac09b770 con 0x7f9dac09aa30 2024-09-17T03:15:00.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.192+0000 7f9db0bc6640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9dac09b8f0 con 0x7f9dac0ce0a0 2024-09-17T03:15:00.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.192+0000 7f9db0bc6640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9dac160ba0 con 0x7f9dac160250 2024-09-17T03:15:00.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.192+0000 7f9daa59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dac09aa30 0x7f9dac06f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:00.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.192+0000 7f9daa59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dac09aa30 0x7f9dac06f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40034/0 (socket says 172.21.15.87:40034) 2024-09-17T03:15:00.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.192+0000 7f9daa59c640 1 -- 172.21.15.87:0/3288485129 learned_addr learned my addr 172.21.15.87:0/3288485129 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:00.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.192+0000 7f9da9d9b640 1 --2- 172.21.15.87:0/3288485129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9dac0ce0a0 0x7f9dac0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:00.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.192+0000 7f9daad9d640 1 --2- 172.21.15.87:0/3288485129 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9dac160250 0x7f9dac15c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:00.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.193+0000 7f9daa59c640 1 -- 172.21.15.87:0/3288485129 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9dac160250 msgr2=0x7f9dac15c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:00.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.193+0000 7f9daa59c640 1 --2- 172.21.15.87:0/3288485129 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9dac160250 0x7f9dac15c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:00.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.193+0000 7f9daa59c640 1 -- 172.21.15.87:0/3288485129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9dac0ce0a0 msgr2=0x7f9dac0c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:00.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.193+0000 7f9daa59c640 1 --2- 172.21.15.87:0/3288485129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9dac0ce0a0 0x7f9dac0c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:00.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.193+0000 7f9daa59c640 1 -- 172.21.15.87:0/3288485129 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dac06f940 con 0x7f9dac09aa30 2024-09-17T03:15:00.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.193+0000 7f9daa59c640 1 --2- 172.21.15.87:0/3288485129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dac09aa30 0x7f9dac06f180 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00d750 tx=0x7f9d9c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:00.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.193+0000 7f9d9affd640 1 -- 172.21.15.87:0/3288485129 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c017070 con 0x7f9dac09aa30 2024-09-17T03:15:00.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.193+0000 7f9d9affd640 1 -- 172.21.15.87:0/3288485129 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d9c009d40 con 0x7f9dac09aa30 2024-09-17T03:15:00.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.193+0000 7f9db0bc6640 1 -- 172.21.15.87:0/3288485129 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dac06fc20 con 0x7f9dac09aa30 2024-09-17T03:15:00.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.193+0000 7f9d9affd640 1 -- 172.21.15.87:0/3288485129 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c0106c0 con 0x7f9dac09aa30 2024-09-17T03:15:00.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.193+0000 7f9db0bc6640 1 -- 172.21.15.87:0/3288485129 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dac157100 con 0x7f9dac09aa30 2024-09-17T03:15:00.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.194+0000 7f9db0bc6640 1 -- 172.21.15.87:0/3288485129 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dac0d0020 con 0x7f9dac09aa30 2024-09-17T03:15:00.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.195+0000 7f9d9affd640 1 -- 172.21.15.87:0/3288485129 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9d9c01a070 con 0x7f9dac09aa30 2024-09-17T03:15:00.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.195+0000 7f9d9affd640 1 --2- 172.21.15.87:0/3288485129 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9d8c078b90 0x7f9d8c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:00.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.195+0000 7f9d9affd640 1 -- 172.21.15.87:0/3288485129 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f9d9c09ad70 con 0x7f9dac09aa30 2024-09-17T03:15:00.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.196+0000 7f9da9d9b640 1 --2- 172.21.15.87:0/3288485129 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9d8c078b90 0x7f9d8c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:00.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.196+0000 7f9da9d9b640 1 --2- 172.21.15.87:0/3288485129 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9d8c078b90 0x7f9d8c07b0a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9dac1f9020 tx=0x7f9da000b260 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:00.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.200+0000 7f9d9affd640 1 -- 172.21.15.87:0/3288485129 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9d9c00de60 con 0x7f9dac09aa30 2024-09-17T03:15:00.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.442+0000 7f9db0bc6640 1 -- 172.21.15.87:0/3288485129 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f9dac0b4a70 con 0x7f9dac09aa30 2024-09-17T03:15:00.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.442+0000 7f9d9affd640 1 -- 172.21.15.87:0/3288485129 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2829 (secure 0 0 0) 0x7f9d9c062bf0 con 0x7f9dac09aa30 2024-09-17T03:15:00.441 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:00.441 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":71,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":34,"num_osds":8,"num_up_osds":8,"osd_up_since":1726542841,"num_in_osds":8,"osd_in_since":1726542832,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":80},{"state_name":"unknown","count":12},{"state_name":"creating+peering","count":5}],"num_pgs":97,"num_pools":3,"num_objects":22,"data_bytes":592778,"bytes_used":229851136,"bytes_avail":772864262144,"bytes_total":773094113280,"unknown_pgs_ratio":0.12371134012937546,"inactive_pgs_ratio":0.051546391099691391,"write_bytes_sec":2801,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":13,"btime":"2024-09-17T03:14:58:337521+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":5004}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-17T03:14:55.426314+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T03:15:00.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.444+0000 7f9db0bc6640 1 -- 172.21.15.87:0/3288485129 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9d8c078b90 msgr2=0x7f9d8c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:00.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.444+0000 7f9db0bc6640 1 --2- 172.21.15.87:0/3288485129 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9d8c078b90 0x7f9d8c07b0a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9dac1f9020 tx=0x7f9da000b260 comp rx=0 tx=0).stop 2024-09-17T03:15:00.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.444+0000 7f9db0bc6640 1 -- 172.21.15.87:0/3288485129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dac09aa30 msgr2=0x7f9dac06f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:00.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.444+0000 7f9db0bc6640 1 --2- 172.21.15.87:0/3288485129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dac09aa30 0x7f9dac06f180 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00d750 tx=0x7f9d9c00dc10 comp rx=0 tx=0).stop 2024-09-17T03:15:00.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.444+0000 7f9db0bc6640 1 -- 172.21.15.87:0/3288485129 shutdown_connections 2024-09-17T03:15:00.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.444+0000 7f9db0bc6640 1 --2- 172.21.15.87:0/3288485129 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9d8c078b90 0x7f9d8c07b0a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:00.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.444+0000 7f9db0bc6640 1 --2- 172.21.15.87:0/3288485129 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9dac160250 0x7f9dac15c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:00.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.444+0000 7f9db0bc6640 1 --2- 172.21.15.87:0/3288485129 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9dac0ce0a0 0x7f9dac0c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:00.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.444+0000 7f9db0bc6640 1 --2- 172.21.15.87:0/3288485129 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dac09aa30 0x7f9dac06f180 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:00.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.444+0000 7f9db0bc6640 1 -- 172.21.15.87:0/3288485129 >> 172.21.15.87:0/3288485129 conn(0x7f9dac0b3d30 msgr2=0x7f9dac0b62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:00.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.445+0000 7f9db0bc6640 1 -- 172.21.15.87:0/3288485129 shutdown_connections 2024-09-17T03:15:00.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.445+0000 7f9db0bc6640 1 -- 172.21.15.87:0/3288485129 wait complete. 2024-09-17T03:15:00.454 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T03:15:00.455 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:15:00.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.568+0000 7f089ac86640 1 -- 172.21.15.87:0/1358433855 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0884003a30 msgr2=0x7f0884003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:00.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.568+0000 7f089ac86640 1 --2- 172.21.15.87:0/1358433855 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0884003a30 0x7f0884003ea0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0888009b50 tx=0x7f0884005830 comp rx=0 tx=0).stop 2024-09-17T03:15:00.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.568+0000 7f0899223640 1 -- 172.21.15.87:0/1358433855 reap_dead start 2024-09-17T03:15:00.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.568+0000 7f089ac86640 1 -- 172.21.15.87:0/1358433855 shutdown_connections 2024-09-17T03:15:00.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.568+0000 7f089ac86640 1 -- 172.21.15.87:0/1358433855 >> 172.21.15.87:0/1358433855 conn(0x7f08940d4470 msgr2=0x7f08940d4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:00.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.568+0000 7f089ac86640 1 -- 172.21.15.87:0/1358433855 shutdown_connections 2024-09-17T03:15:00.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.568+0000 7f089ac86640 1 -- 172.21.15.87:0/1358433855 wait complete. 2024-09-17T03:15:00.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.570+0000 7f089ac86640 1 Processor -- start 2024-09-17T03:15:00.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.570+0000 7f089ac86640 1 -- start start 2024-09-17T03:15:00.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.571+0000 7f089ac86640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0884003a30 0x7f0894159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:00.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.571+0000 7f089ac86640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08940cc130 0x7f08940cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:00.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.571+0000 7f089ac86640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f08940cca80 0x7f08941f4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:00.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.571+0000 7f089ac86640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f089415f280 con 0x7f08940cc130 2024-09-17T03:15:00.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.571+0000 7f089ac86640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f089415f100 con 0x7f0884003a30 2024-09-17T03:15:00.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.571+0000 7f089ac86640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f089415f400 con 0x7f08940cca80 2024-09-17T03:15:00.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.571+0000 7f0893fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08940cc130 0x7f08940cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:00.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.571+0000 7f0893fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08940cc130 0x7f08940cc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40046/0 (socket says 172.21.15.87:40046) 2024-09-17T03:15:00.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.571+0000 7f0893fff640 1 -- 172.21.15.87:0/3301136660 learned_addr learned my addr 172.21.15.87:0/3301136660 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:00.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.571+0000 7f0898a22640 1 --2- 172.21.15.87:0/3301136660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0884003a30 0x7f0894159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:00.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.571+0000 7f0899223640 1 --2- 172.21.15.87:0/3301136660 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f08940cca80 0x7f08941f4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:00.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.572+0000 7f0893fff640 1 -- 172.21.15.87:0/3301136660 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f08940cca80 msgr2=0x7f08941f4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:00.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.572+0000 7f0893fff640 1 --2- 172.21.15.87:0/3301136660 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f08940cca80 0x7f08941f4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:00.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.572+0000 7f0893fff640 1 -- 172.21.15.87:0/3301136660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0884003a30 msgr2=0x7f0894159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:00.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.572+0000 7f0893fff640 1 --2- 172.21.15.87:0/3301136660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0884003a30 0x7f0894159a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:00.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.572+0000 7f0893fff640 1 -- 172.21.15.87:0/3301136660 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0894159f80 con 0x7f08940cc130 2024-09-17T03:15:00.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.572+0000 7f0893fff640 1 --2- 172.21.15.87:0/3301136660 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08940cc130 0x7f08940cc540 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0880003fb0 tx=0x7f088000f7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:00.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.572+0000 7f08917fa640 1 -- 172.21.15.87:0/3301136660 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0880015070 con 0x7f08940cc130 2024-09-17T03:15:00.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.572+0000 7f08917fa640 1 -- 172.21.15.87:0/3301136660 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0880011900 con 0x7f08940cc130 2024-09-17T03:15:00.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.572+0000 7f089ac86640 1 -- 172.21.15.87:0/3301136660 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08940c7f10 con 0x7f08940cc130 2024-09-17T03:15:00.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.572+0000 7f08917fa640 1 -- 172.21.15.87:0/3301136660 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0880011d50 con 0x7f08940cc130 2024-09-17T03:15:00.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.572+0000 7f089ac86640 1 -- 172.21.15.87:0/3301136660 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08940cb650 con 0x7f08940cc130 2024-09-17T03:15:00.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.573+0000 7f089ac86640 1 -- 172.21.15.87:0/3301136660 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08940ca830 con 0x7f08940cc130 2024-09-17T03:15:00.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.574+0000 7f08917fa640 1 -- 172.21.15.87:0/3301136660 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f08800103c0 con 0x7f08940cc130 2024-09-17T03:15:00.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.574+0000 7f08917fa640 1 --2- 172.21.15.87:0/3301136660 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0878078b90 0x7f087807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:00.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.574+0000 7f08917fa640 1 -- 172.21.15.87:0/3301136660 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f088009b7e0 con 0x7f08940cc130 2024-09-17T03:15:00.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.575+0000 7f0898a22640 1 --2- 172.21.15.87:0/3301136660 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0878078b90 0x7f087807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:00.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.575+0000 7f0898a22640 1 --2- 172.21.15.87:0/3301136660 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0878078b90 0x7f087807b0a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0888035000 tx=0x7f088802d040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:00.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.579+0000 7f08917fa640 1 -- 172.21.15.87:0/3301136660 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0894159f80 con 0x7f08940cc130 2024-09-17T03:15:00.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.748+0000 7f089ac86640 1 -- 172.21.15.87:0/3301136660 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f08940d2c30 con 0x7f0878078b90 2024-09-17T03:15:00.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.752+0000 7f08917fa640 1 -- 172.21.15.87:0/3301136660 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249913 (secure 0 0 0) 0x7f08940d2c30 con 0x7f0878078b90 2024-09-17T03:15:00.753 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:00.754 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-09-17T03:15:00.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.755+0000 7f089ac86640 1 -- 172.21.15.87:0/3301136660 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0878078b90 msgr2=0x7f087807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:00.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.755+0000 7f089ac86640 1 --2- 172.21.15.87:0/3301136660 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0878078b90 0x7f087807b0a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0888035000 tx=0x7f088802d040 comp rx=0 tx=0).stop 2024-09-17T03:15:00.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.755+0000 7f089ac86640 1 -- 172.21.15.87:0/3301136660 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08940cc130 msgr2=0x7f08940cc540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:00.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.755+0000 7f089ac86640 1 --2- 172.21.15.87:0/3301136660 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08940cc130 0x7f08940cc540 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0880003fb0 tx=0x7f088000f7e0 comp rx=0 tx=0).stop 2024-09-17T03:15:00.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.755+0000 7f089ac86640 1 -- 172.21.15.87:0/3301136660 shutdown_connections 2024-09-17T03:15:00.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.755+0000 7f089ac86640 1 --2- 172.21.15.87:0/3301136660 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0878078b90 0x7f087807b0a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:00.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.755+0000 7f089ac86640 1 --2- 172.21.15.87:0/3301136660 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f08940cca80 0x7f08941f4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:00.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.755+0000 7f089ac86640 1 --2- 172.21.15.87:0/3301136660 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08940cc130 0x7f08940cc540 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:00.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.755+0000 7f089ac86640 1 --2- 172.21.15.87:0/3301136660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0884003a30 0x7f0894159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:00.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.755+0000 7f089ac86640 1 -- 172.21.15.87:0/3301136660 >> 172.21.15.87:0/3301136660 conn(0x7f08940d4470 msgr2=0x7f089415d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:00.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.755+0000 7f089ac86640 1 -- 172.21.15.87:0/3301136660 shutdown_connections 2024-09-17T03:15:00.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:00.755+0000 7f089ac86640 1 -- 172.21.15.87:0/3301136660 wait complete. 2024-09-17T03:15:00.765 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-09-17T03:14:59.427221+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":142,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":170,"acting":170,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":94,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224464,"kb_used_data":4020,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750256,"statfs":{"total":773094113280,"available":772864262144,"internally_reserved":0,"allocated":4116480,"data_stored":2277111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.117237"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351505+0000","last_change":"2024-09-17T03:14:56.344756+0000","last_active":"2024-09-17T03:14:56.351505+0000","last_peered":"2024-09-17T03:14:56.351505+0000","last_clean":"2024-09-17T03:14:56.351505+0000","last_became_active":"2024-09-17T03:14:56.343497+0000","last_became_peered":"2024-09-17T03:14:56.343497+0000","last_unstale":"2024-09-17T03:14:56.351505+0000","last_undegraded":"2024-09-17T03:14:56.351505+0000","last_fullsized":"2024-09-17T03:14:56.351505+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:07.377313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344120+0000","last_change":"2024-09-17T03:14:56.348503+0000","last_active":"2024-09-17T03:14:57.344120+0000","last_peered":"2024-09-17T03:14:57.344120+0000","last_clean":"2024-09-17T03:14:57.344120+0000","last_became_active":"2024-09-17T03:14:56.348172+0000","last_became_peered":"2024-09-17T03:14:56.348172+0000","last_unstale":"2024-09-17T03:14:57.344120+0000","last_undegraded":"2024-09-17T03:14:57.344120+0000","last_fullsized":"2024-09-17T03:14:57.344120+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:52:09.125634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.349842+0000","last_change":"2024-09-17T03:14:56.347013+0000","last_active":"2024-09-17T03:14:56.349842+0000","last_peered":"2024-09-17T03:14:56.349842+0000","last_clean":"2024-09-17T03:14:56.349842+0000","last_became_active":"2024-09-17T03:14:56.346718+0000","last_became_peered":"2024-09-17T03:14:56.346718+0000","last_unstale":"2024-09-17T03:14:56.349842+0000","last_undegraded":"2024-09-17T03:14:56.349842+0000","last_fullsized":"2024-09-17T03:14:56.349842+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:42:17.382050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.343763+0000","last_change":"2024-09-17T03:14:56.344528+0000","last_active":"2024-09-17T03:14:57.343763+0000","last_peered":"2024-09-17T03:14:57.343763+0000","last_clean":"2024-09-17T03:14:57.343763+0000","last_became_active":"2024-09-17T03:14:56.344226+0000","last_became_peered":"2024-09-17T03:14:56.344226+0000","last_unstale":"2024-09-17T03:14:57.343763+0000","last_undegraded":"2024-09-17T03:14:57.343763+0000","last_fullsized":"2024-09-17T03:14:57.343763+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:09:48.039023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.340976+0000","last_change":"2024-09-17T03:14:56.342382+0000","last_active":"2024-09-17T03:14:57.340976+0000","last_peered":"2024-09-17T03:14:57.340976+0000","last_clean":"2024-09-17T03:14:57.340976+0000","last_became_active":"2024-09-17T03:14:56.341980+0000","last_became_peered":"2024-09-17T03:14:56.341980+0000","last_unstale":"2024-09-17T03:14:57.340976+0000","last_undegraded":"2024-09-17T03:14:57.340976+0000","last_fullsized":"2024-09-17T03:14:57.340976+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:11.714679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.345446+0000","last_change":"2024-09-17T03:14:56.343019+0000","last_active":"2024-09-17T03:14:56.345446+0000","last_peered":"2024-09-17T03:14:56.345446+0000","last_clean":"2024-09-17T03:14:56.345446+0000","last_became_active":"2024-09-17T03:14:56.342705+0000","last_became_peered":"2024-09-17T03:14:56.342705+0000","last_unstale":"2024-09-17T03:14:56.345446+0000","last_undegraded":"2024-09-17T03:14:56.345446+0000","last_fullsized":"2024-09-17T03:14:56.345446+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:04.002930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.352770+0000","last_change":"2024-09-17T03:14:56.346692+0000","last_active":"2024-09-17T03:14:56.352770+0000","last_peered":"2024-09-17T03:14:56.352770+0000","last_clean":"2024-09-17T03:14:56.352770+0000","last_became_active":"2024-09-17T03:14:56.346397+0000","last_became_peered":"2024-09-17T03:14:56.346397+0000","last_unstale":"2024-09-17T03:14:56.352770+0000","last_undegraded":"2024-09-17T03:14:56.352770+0000","last_fullsized":"2024-09-17T03:14:56.352770+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:28.743694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.344785+0000","last_change":"2024-09-17T03:14:56.339990+0000","last_active":"2024-09-17T03:14:56.344785+0000","last_peered":"2024-09-17T03:14:56.344785+0000","last_clean":"2024-09-17T03:14:56.344785+0000","last_became_active":"2024-09-17T03:14:56.339281+0000","last_became_peered":"2024-09-17T03:14:56.339281+0000","last_unstale":"2024-09-17T03:14:56.344785+0000","last_undegraded":"2024-09-17T03:14:56.344785+0000","last_fullsized":"2024-09-17T03:14:56.344785+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:32:19.770373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351104+0000","last_change":"2024-09-17T03:14:56.342266+0000","last_active":"2024-09-17T03:14:56.351104+0000","last_peered":"2024-09-17T03:14:56.351104+0000","last_clean":"2024-09-17T03:14:56.351104+0000","last_became_active":"2024-09-17T03:14:56.341978+0000","last_became_peered":"2024-09-17T03:14:56.341978+0000","last_unstale":"2024-09-17T03:14:56.351104+0000","last_undegraded":"2024-09-17T03:14:56.351104+0000","last_fullsized":"2024-09-17T03:14:56.351104+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:50:28.343075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.345419+0000","last_change":"2024-09-17T03:14:56.342630+0000","last_active":"2024-09-17T03:14:56.345419+0000","last_peered":"2024-09-17T03:14:56.345419+0000","last_clean":"2024-09-17T03:14:56.345419+0000","last_became_active":"2024-09-17T03:14:56.342175+0000","last_became_peered":"2024-09-17T03:14:56.342175+0000","last_unstale":"2024-09-17T03:14:56.345419+0000","last_undegraded":"2024-09-17T03:14:56.345419+0000","last_fullsized":"2024-09-17T03:14:56.345419+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:02:14.647806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.357637+0000","last_change":"2024-09-17T03:14:56.351892+0000","last_active":"2024-09-17T03:14:56.357637+0000","last_peered":"2024-09-17T03:14:56.357637+0000","last_clean":"2024-09-17T03:14:56.357637+0000","last_became_active":"2024-09-17T03:14:56.351149+0000","last_became_peered":"2024-09-17T03:14:56.351149+0000","last_unstale":"2024-09-17T03:14:56.357637+0000","last_undegraded":"2024-09-17T03:14:56.357637+0000","last_fullsized":"2024-09-17T03:14:56.357637+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:14:16.498828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351551+0000","last_change":"2024-09-17T03:14:56.349422+0000","last_active":"2024-09-17T03:14:56.351551+0000","last_peered":"2024-09-17T03:14:56.351551+0000","last_clean":"2024-09-17T03:14:56.351551+0000","last_became_active":"2024-09-17T03:14:56.349132+0000","last_became_peered":"2024-09-17T03:14:56.349132+0000","last_unstale":"2024-09-17T03:14:56.351551+0000","last_undegraded":"2024-09-17T03:14:56.351551+0000","last_fullsized":"2024-09-17T03:14:56.351551+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:27.164191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.340907+0000","last_change":"2024-09-17T03:14:56.337002+0000","last_active":"2024-09-17T03:14:56.340907+0000","last_peered":"2024-09-17T03:14:56.340907+0000","last_clean":"2024-09-17T03:14:56.340907+0000","last_became_active":"2024-09-17T03:14:56.336649+0000","last_became_peered":"2024-09-17T03:14:56.336649+0000","last_unstale":"2024-09-17T03:14:56.340907+0000","last_undegraded":"2024-09-17T03:14:56.340907+0000","last_fullsized":"2024-09-17T03:14:56.340907+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:02:48.048417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.354704+0000","last_change":"2024-09-17T03:14:56.346405+0000","last_active":"2024-09-17T03:14:56.354704+0000","last_peered":"2024-09-17T03:14:56.354704+0000","last_clean":"2024-09-17T03:14:56.354704+0000","last_became_active":"2024-09-17T03:14:56.346037+0000","last_became_peered":"2024-09-17T03:14:56.346037+0000","last_unstale":"2024-09-17T03:14:56.354704+0000","last_undegraded":"2024-09-17T03:14:56.354704+0000","last_fullsized":"2024-09-17T03:14:56.354704+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:04:18.618725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341208+0000","last_change":"2024-09-17T03:14:56.345147+0000","last_active":"2024-09-17T03:14:57.341208+0000","last_peered":"2024-09-17T03:14:57.341208+0000","last_clean":"2024-09-17T03:14:57.341208+0000","last_became_active":"2024-09-17T03:14:56.344844+0000","last_became_peered":"2024-09-17T03:14:56.344844+0000","last_unstale":"2024-09-17T03:14:57.341208+0000","last_undegraded":"2024-09-17T03:14:57.341208+0000","last_fullsized":"2024-09-17T03:14:57.341208+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:30:40.186081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341134+0000","last_change":"2024-09-17T03:14:56.339650+0000","last_active":"2024-09-17T03:14:57.341134+0000","last_peered":"2024-09-17T03:14:57.341134+0000","last_clean":"2024-09-17T03:14:57.341134+0000","last_became_active":"2024-09-17T03:14:56.338719+0000","last_became_peered":"2024-09-17T03:14:56.338719+0000","last_unstale":"2024-09-17T03:14:57.341134+0000","last_undegraded":"2024-09-17T03:14:57.341134+0000","last_fullsized":"2024-09-17T03:14:57.341134+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:51:33.167537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351809+0000","last_change":"2024-09-17T03:14:56.341270+0000","last_active":"2024-09-17T03:14:56.351809+0000","last_peered":"2024-09-17T03:14:56.351809+0000","last_clean":"2024-09-17T03:14:56.351809+0000","last_became_active":"2024-09-17T03:14:56.340876+0000","last_became_peered":"2024-09-17T03:14:56.340876+0000","last_unstale":"2024-09-17T03:14:56.351809+0000","last_undegraded":"2024-09-17T03:14:56.351809+0000","last_fullsized":"2024-09-17T03:14:56.351809+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:37:07.377313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.349912+0000","last_change":"2024-09-17T03:14:56.344410+0000","last_active":"2024-09-17T03:14:56.349912+0000","last_peered":"2024-09-17T03:14:56.349912+0000","last_clean":"2024-09-17T03:14:56.349912+0000","last_became_active":"2024-09-17T03:14:56.344151+0000","last_became_peered":"2024-09-17T03:14:56.344151+0000","last_unstale":"2024-09-17T03:14:56.349912+0000","last_undegraded":"2024-09-17T03:14:56.349912+0000","last_fullsized":"2024-09-17T03:14:56.349912+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:38:12.850097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.356153+0000","last_change":"2024-09-17T03:14:56.354343+0000","last_active":"2024-09-17T03:14:56.356153+0000","last_peered":"2024-09-17T03:14:56.356153+0000","last_clean":"2024-09-17T03:14:56.356153+0000","last_became_active":"2024-09-17T03:14:56.354020+0000","last_became_peered":"2024-09-17T03:14:56.354020+0000","last_unstale":"2024-09-17T03:14:56.356153+0000","last_undegraded":"2024-09-17T03:14:56.356153+0000","last_fullsized":"2024-09-17T03:14:56.356153+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:07:23.062711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351758+0000","last_change":"2024-09-17T03:14:56.340466+0000","last_active":"2024-09-17T03:14:56.351758+0000","last_peered":"2024-09-17T03:14:56.351758+0000","last_clean":"2024-09-17T03:14:56.351758+0000","last_became_active":"2024-09-17T03:14:56.340135+0000","last_became_peered":"2024-09-17T03:14:56.340135+0000","last_unstale":"2024-09-17T03:14:56.351758+0000","last_undegraded":"2024-09-17T03:14:56.351758+0000","last_fullsized":"2024-09-17T03:14:56.351758+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:05.079987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.356113+0000","last_change":"2024-09-17T03:14:56.353500+0000","last_active":"2024-09-17T03:14:56.356113+0000","last_peered":"2024-09-17T03:14:56.356113+0000","last_clean":"2024-09-17T03:14:56.356113+0000","last_became_active":"2024-09-17T03:14:56.353211+0000","last_became_peered":"2024-09-17T03:14:56.353211+0000","last_unstale":"2024-09-17T03:14:56.356113+0000","last_undegraded":"2024-09-17T03:14:56.356113+0000","last_fullsized":"2024-09-17T03:14:56.356113+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:04:11.473585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.343839+0000","last_change":"2024-09-17T03:14:56.329704+0000","last_active":"2024-09-17T03:14:57.343839+0000","last_peered":"2024-09-17T03:14:57.343839+0000","last_clean":"2024-09-17T03:14:57.343839+0000","last_became_active":"2024-09-17T03:14:56.329354+0000","last_became_peered":"2024-09-17T03:14:56.329354+0000","last_unstale":"2024-09-17T03:14:57.343839+0000","last_undegraded":"2024-09-17T03:14:57.343839+0000","last_fullsized":"2024-09-17T03:14:57.343839+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:59:50.923956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.352937+0000","last_change":"2024-09-17T03:14:56.348309+0000","last_active":"2024-09-17T03:14:56.352937+0000","last_peered":"2024-09-17T03:14:56.352937+0000","last_clean":"2024-09-17T03:14:56.352937+0000","last_became_active":"2024-09-17T03:14:56.347990+0000","last_became_peered":"2024-09-17T03:14:56.347990+0000","last_unstale":"2024-09-17T03:14:56.352937+0000","last_undegraded":"2024-09-17T03:14:56.352937+0000","last_fullsized":"2024-09-17T03:14:56.352937+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:25.974302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.348883+0000","last_change":"2024-09-17T03:14:56.342697+0000","last_active":"2024-09-17T03:14:56.348883+0000","last_peered":"2024-09-17T03:14:56.348883+0000","last_clean":"2024-09-17T03:14:56.348883+0000","last_became_active":"2024-09-17T03:14:56.342375+0000","last_became_peered":"2024-09-17T03:14:56.342375+0000","last_unstale":"2024-09-17T03:14:56.348883+0000","last_undegraded":"2024-09-17T03:14:56.348883+0000","last_fullsized":"2024-09-17T03:14:56.348883+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:04.426421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.350990+0000","last_change":"2024-09-17T03:14:56.346711+0000","last_active":"2024-09-17T03:14:56.350990+0000","last_peered":"2024-09-17T03:14:56.350990+0000","last_clean":"2024-09-17T03:14:56.350990+0000","last_became_active":"2024-09-17T03:14:56.346399+0000","last_became_peered":"2024-09-17T03:14:56.346399+0000","last_unstale":"2024-09-17T03:14:56.350990+0000","last_undegraded":"2024-09-17T03:14:56.350990+0000","last_fullsized":"2024-09-17T03:14:56.350990+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:20.900685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.352810+0000","last_change":"2024-09-17T03:14:56.348432+0000","last_active":"2024-09-17T03:14:56.352810+0000","last_peered":"2024-09-17T03:14:56.352810+0000","last_clean":"2024-09-17T03:14:56.352810+0000","last_became_active":"2024-09-17T03:14:56.348048+0000","last_became_peered":"2024-09-17T03:14:56.348048+0000","last_unstale":"2024-09-17T03:14:56.352810+0000","last_undegraded":"2024-09-17T03:14:56.352810+0000","last_fullsized":"2024-09-17T03:14:56.352810+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:26:13.257738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351063+0000","last_change":"2024-09-17T03:14:56.346523+0000","last_active":"2024-09-17T03:14:56.351063+0000","last_peered":"2024-09-17T03:14:56.351063+0000","last_clean":"2024-09-17T03:14:56.351063+0000","last_became_active":"2024-09-17T03:14:56.346207+0000","last_became_peered":"2024-09-17T03:14:56.346207+0000","last_unstale":"2024-09-17T03:14:56.351063+0000","last_undegraded":"2024-09-17T03:14:56.351063+0000","last_fullsized":"2024-09-17T03:14:56.351063+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:53.559531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351860+0000","last_change":"2024-09-17T03:14:56.341373+0000","last_active":"2024-09-17T03:14:56.351860+0000","last_peered":"2024-09-17T03:14:56.351860+0000","last_clean":"2024-09-17T03:14:56.351860+0000","last_became_active":"2024-09-17T03:14:56.341066+0000","last_became_peered":"2024-09-17T03:14:56.341066+0000","last_unstale":"2024-09-17T03:14:56.351860+0000","last_undegraded":"2024-09-17T03:14:56.351860+0000","last_fullsized":"2024-09-17T03:14:56.351860+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:28.743694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.354608+0000","last_change":"2024-09-17T03:14:55.321527+0000","last_active":"2024-09-17T03:14:56.354608+0000","last_peered":"2024-09-17T03:14:56.354608+0000","last_clean":"2024-09-17T03:14:56.354608+0000","last_became_active":"2024-09-17T03:14:55.320714+0000","last_became_peered":"2024-09-17T03:14:55.320714+0000","last_unstale":"2024-09-17T03:14:56.354608+0000","last_undegraded":"2024-09-17T03:14:56.354608+0000","last_fullsized":"2024-09-17T03:14:56.354608+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:16:27.821016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351470+0000","last_change":"2024-09-17T03:14:56.344463+0000","last_active":"2024-09-17T03:14:56.351470+0000","last_peered":"2024-09-17T03:14:56.351470+0000","last_clean":"2024-09-17T03:14:56.351470+0000","last_became_active":"2024-09-17T03:14:56.344125+0000","last_became_peered":"2024-09-17T03:14:56.344125+0000","last_unstale":"2024-09-17T03:14:56.351470+0000","last_undegraded":"2024-09-17T03:14:56.351470+0000","last_fullsized":"2024-09-17T03:14:56.351470+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:36:05.079987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T03:14:54.312526+0000","last_change":"2024-09-17T03:14:54.312526+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.312526+0000","last_undegraded":"2024-09-17T03:14:54.312526+0000","last_fullsized":"2024-09-17T03:14:54.312526+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.356182+0000","last_change":"2024-09-17T03:14:56.354666+0000","last_active":"2024-09-17T03:14:56.356182+0000","last_peered":"2024-09-17T03:14:56.356182+0000","last_clean":"2024-09-17T03:14:56.356182+0000","last_became_active":"2024-09-17T03:14:56.354360+0000","last_became_peered":"2024-09-17T03:14:56.354360+0000","last_unstale":"2024-09-17T03:14:56.356182+0000","last_undegraded":"2024-09-17T03:14:56.356182+0000","last_fullsized":"2024-09-17T03:14:56.356182+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:19.990834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341309+0000","last_change":"2024-09-17T03:14:55.332084+0000","last_active":"2024-09-17T03:14:57.341309+0000","last_peered":"2024-09-17T03:14:57.341309+0000","last_clean":"2024-09-17T03:14:57.341309+0000","last_became_active":"2024-09-17T03:14:55.331735+0000","last_became_peered":"2024-09-17T03:14:55.331735+0000","last_unstale":"2024-09-17T03:14:57.341309+0000","last_undegraded":"2024-09-17T03:14:57.341309+0000","last_fullsized":"2024-09-17T03:14:57.341309+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:23.002081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T03:14:54.312074+0000","last_change":"2024-09-17T03:14:54.312074+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.312074+0000","last_undegraded":"2024-09-17T03:14:54.312074+0000","last_fullsized":"2024-09-17T03:14:54.312074+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.348195+0000","last_change":"2024-09-17T03:14:56.345924+0000","last_active":"2024-09-17T03:14:56.348195+0000","last_peered":"2024-09-17T03:14:56.348195+0000","last_clean":"2024-09-17T03:14:56.348195+0000","last_became_active":"2024-09-17T03:14:56.345340+0000","last_became_peered":"2024-09-17T03:14:56.345340+0000","last_unstale":"2024-09-17T03:14:56.348195+0000","last_undegraded":"2024-09-17T03:14:56.348195+0000","last_fullsized":"2024-09-17T03:14:56.348195+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:39:18.485654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.357022+0000","last_change":"2024-09-17T03:14:55.334211+0000","last_active":"2024-09-17T03:14:56.357022+0000","last_peered":"2024-09-17T03:14:56.357022+0000","last_clean":"2024-09-17T03:14:56.357022+0000","last_became_active":"2024-09-17T03:14:55.333714+0000","last_became_peered":"2024-09-17T03:14:55.333714+0000","last_unstale":"2024-09-17T03:14:56.357022+0000","last_undegraded":"2024-09-17T03:14:56.357022+0000","last_fullsized":"2024-09-17T03:14:56.357022+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:30:39.180289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.353185+0000","last_change":"2024-09-17T03:14:56.348919+0000","last_active":"2024-09-17T03:14:56.353185+0000","last_peered":"2024-09-17T03:14:56.353185+0000","last_clean":"2024-09-17T03:14:56.353185+0000","last_became_active":"2024-09-17T03:14:56.348235+0000","last_became_peered":"2024-09-17T03:14:56.348235+0000","last_unstale":"2024-09-17T03:14:56.353185+0000","last_undegraded":"2024-09-17T03:14:56.353185+0000","last_fullsized":"2024-09-17T03:14:56.353185+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:14:16.498828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.357606+0000","last_change":"2024-09-17T03:14:55.335181+0000","last_active":"2024-09-17T03:14:56.357606+0000","last_peered":"2024-09-17T03:14:56.357606+0000","last_clean":"2024-09-17T03:14:56.357606+0000","last_became_active":"2024-09-17T03:14:55.334860+0000","last_became_peered":"2024-09-17T03:14:55.334860+0000","last_unstale":"2024-09-17T03:14:56.357606+0000","last_undegraded":"2024-09-17T03:14:56.357606+0000","last_fullsized":"2024-09-17T03:14:56.357606+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:34:41.387404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.341334+0000","last_change":"2024-09-17T03:14:56.338147+0000","last_active":"2024-09-17T03:14:56.341334+0000","last_peered":"2024-09-17T03:14:56.341334+0000","last_clean":"2024-09-17T03:14:56.341334+0000","last_became_active":"2024-09-17T03:14:56.337801+0000","last_became_peered":"2024-09-17T03:14:56.337801+0000","last_unstale":"2024-09-17T03:14:56.341334+0000","last_undegraded":"2024-09-17T03:14:56.341334+0000","last_fullsized":"2024-09-17T03:14:56.341334+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:24.007873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.340815+0000","last_change":"2024-09-17T03:14:55.328132+0000","last_active":"2024-09-17T03:14:57.340815+0000","last_peered":"2024-09-17T03:14:57.340815+0000","last_clean":"2024-09-17T03:14:57.340815+0000","last_became_active":"2024-09-17T03:14:55.327773+0000","last_became_peered":"2024-09-17T03:14:55.327773+0000","last_unstale":"2024-09-17T03:14:57.340815+0000","last_undegraded":"2024-09-17T03:14:57.340815+0000","last_fullsized":"2024-09-17T03:14:57.340815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:09.899391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.330123+0000","last_change":"2024-09-17T03:14:55.334322+0000","last_active":"2024-09-17T03:14:56.330123+0000","last_peered":"2024-09-17T03:14:56.330123+0000","last_clean":"2024-09-17T03:14:56.330123+0000","last_became_active":"2024-09-17T03:14:55.333947+0000","last_became_peered":"2024-09-17T03:14:55.333947+0000","last_unstale":"2024-09-17T03:14:56.330123+0000","last_undegraded":"2024-09-17T03:14:56.330123+0000","last_fullsized":"2024-09-17T03:14:56.330123+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:44.089548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351133+0000","last_change":"2024-09-17T03:14:56.345419+0000","last_active":"2024-09-17T03:14:56.351133+0000","last_peered":"2024-09-17T03:14:56.351133+0000","last_clean":"2024-09-17T03:14:56.351133+0000","last_became_active":"2024-09-17T03:14:56.343640+0000","last_became_peered":"2024-09-17T03:14:56.343640+0000","last_unstale":"2024-09-17T03:14:56.351133+0000","last_undegraded":"2024-09-17T03:14:56.351133+0000","last_fullsized":"2024-09-17T03:14:56.351133+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:51:20.330393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344278+0000","last_change":"2024-09-17T03:14:55.323288+0000","last_active":"2024-09-17T03:14:57.344278+0000","last_peered":"2024-09-17T03:14:57.344278+0000","last_clean":"2024-09-17T03:14:57.344278+0000","last_became_active":"2024-09-17T03:14:55.322723+0000","last_became_peered":"2024-09-17T03:14:55.322723+0000","last_unstale":"2024-09-17T03:14:57.344278+0000","last_undegraded":"2024-09-17T03:14:57.344278+0000","last_fullsized":"2024-09-17T03:14:57.344278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:11:24.012880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344030+0000","last_change":"2024-09-17T03:14:55.325887+0000","last_active":"2024-09-17T03:14:57.344030+0000","last_peered":"2024-09-17T03:14:57.344030+0000","last_clean":"2024-09-17T03:14:57.344030+0000","last_became_active":"2024-09-17T03:14:55.325563+0000","last_became_peered":"2024-09-17T03:14:55.325563+0000","last_unstale":"2024-09-17T03:14:57.344030+0000","last_undegraded":"2024-09-17T03:14:57.344030+0000","last_fullsized":"2024-09-17T03:14:57.344030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:22.517631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.354205+0000","last_change":"2024-09-17T03:14:55.329954+0000","last_active":"2024-09-17T03:14:56.354205+0000","last_peered":"2024-09-17T03:14:56.354205+0000","last_clean":"2024-09-17T03:14:56.354205+0000","last_became_active":"2024-09-17T03:14:55.329631+0000","last_became_peered":"2024-09-17T03:14:55.329631+0000","last_unstale":"2024-09-17T03:14:56.354205+0000","last_undegraded":"2024-09-17T03:14:56.354205+0000","last_fullsized":"2024-09-17T03:14:56.354205+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:45.028465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.354624+0000","last_change":"2024-09-17T03:14:56.352353+0000","last_active":"2024-09-17T03:14:56.354624+0000","last_peered":"2024-09-17T03:14:56.354624+0000","last_clean":"2024-09-17T03:14:56.354624+0000","last_became_active":"2024-09-17T03:14:56.352041+0000","last_became_peered":"2024-09-17T03:14:56.352041+0000","last_unstale":"2024-09-17T03:14:56.354624+0000","last_undegraded":"2024-09-17T03:14:56.354624+0000","last_fullsized":"2024-09-17T03:14:56.354624+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:43:07.926923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.360502+0000","last_change":"2024-09-17T03:14:55.332350+0000","last_active":"2024-09-17T03:14:56.360502+0000","last_peered":"2024-09-17T03:14:56.360502+0000","last_clean":"2024-09-17T03:14:56.360502+0000","last_became_active":"2024-09-17T03:14:55.332010+0000","last_became_peered":"2024-09-17T03:14:55.332010+0000","last_unstale":"2024-09-17T03:14:56.360502+0000","last_undegraded":"2024-09-17T03:14:56.360502+0000","last_fullsized":"2024-09-17T03:14:56.360502+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:19.084170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351935+0000","last_change":"2024-09-17T03:14:56.344117+0000","last_active":"2024-09-17T03:14:56.351935+0000","last_peered":"2024-09-17T03:14:56.351935+0000","last_clean":"2024-09-17T03:14:56.351935+0000","last_became_active":"2024-09-17T03:14:56.343821+0000","last_became_peered":"2024-09-17T03:14:56.343821+0000","last_unstale":"2024-09-17T03:14:56.351935+0000","last_undegraded":"2024-09-17T03:14:56.351935+0000","last_fullsized":"2024-09-17T03:14:56.351935+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:25.974302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.355844+0000","last_change":"2024-09-17T03:14:55.333495+0000","last_active":"2024-09-17T03:14:56.355844+0000","last_peered":"2024-09-17T03:14:56.355844+0000","last_clean":"2024-09-17T03:14:56.355844+0000","last_became_active":"2024-09-17T03:14:55.332779+0000","last_became_peered":"2024-09-17T03:14:55.332779+0000","last_unstale":"2024-09-17T03:14:56.355844+0000","last_undegraded":"2024-09-17T03:14:56.355844+0000","last_fullsized":"2024-09-17T03:14:56.355844+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:19.084170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T03:14:54.311860+0000","last_change":"2024-09-17T03:14:54.311860+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.311860+0000","last_undegraded":"2024-09-17T03:14:54.311860+0000","last_fullsized":"2024-09-17T03:14:54.311860+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.345568+0000","last_change":"2024-09-17T03:14:56.341404+0000","last_active":"2024-09-17T03:14:56.345568+0000","last_peered":"2024-09-17T03:14:56.345568+0000","last_clean":"2024-09-17T03:14:56.345568+0000","last_became_active":"2024-09-17T03:14:56.341086+0000","last_became_peered":"2024-09-17T03:14:56.341086+0000","last_unstale":"2024-09-17T03:14:56.345568+0000","last_undegraded":"2024-09-17T03:14:56.345568+0000","last_fullsized":"2024-09-17T03:14:56.345568+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:19:29.104716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T03:14:54.313012+0000","last_change":"2024-09-17T03:14:54.313012+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.313012+0000","last_undegraded":"2024-09-17T03:14:54.313012+0000","last_fullsized":"2024-09-17T03:14:54.313012+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351204+0000","last_change":"2024-09-17T03:14:56.347429+0000","last_active":"2024-09-17T03:14:56.351204+0000","last_peered":"2024-09-17T03:14:56.351204+0000","last_clean":"2024-09-17T03:14:56.351204+0000","last_became_active":"2024-09-17T03:14:56.347101+0000","last_became_peered":"2024-09-17T03:14:56.347101+0000","last_unstale":"2024-09-17T03:14:56.351204+0000","last_undegraded":"2024-09-17T03:14:56.351204+0000","last_fullsized":"2024-09-17T03:14:56.351204+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:04:46.793182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.340674+0000","last_change":"2024-09-17T03:14:55.330923+0000","last_active":"2024-09-17T03:14:57.340674+0000","last_peered":"2024-09-17T03:14:57.340674+0000","last_clean":"2024-09-17T03:14:57.340674+0000","last_became_active":"2024-09-17T03:14:55.330601+0000","last_became_peered":"2024-09-17T03:14:55.330601+0000","last_unstale":"2024-09-17T03:14:57.340674+0000","last_undegraded":"2024-09-17T03:14:57.340674+0000","last_fullsized":"2024-09-17T03:14:57.340674+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:21.326711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351107+0000","last_change":"2024-09-17T03:14:56.343137+0000","last_active":"2024-09-17T03:14:56.351107+0000","last_peered":"2024-09-17T03:14:56.351107+0000","last_clean":"2024-09-17T03:14:56.351107+0000","last_became_active":"2024-09-17T03:14:56.342838+0000","last_became_peered":"2024-09-17T03:14:56.342838+0000","last_unstale":"2024-09-17T03:14:56.351107+0000","last_undegraded":"2024-09-17T03:14:56.351107+0000","last_fullsized":"2024-09-17T03:14:56.351107+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:54:34.725205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341150+0000","last_change":"2024-09-17T03:14:55.331984+0000","last_active":"2024-09-17T03:14:57.341150+0000","last_peered":"2024-09-17T03:14:57.341150+0000","last_clean":"2024-09-17T03:14:57.341150+0000","last_became_active":"2024-09-17T03:14:55.331543+0000","last_became_peered":"2024-09-17T03:14:55.331543+0000","last_unstale":"2024-09-17T03:14:57.341150+0000","last_undegraded":"2024-09-17T03:14:57.341150+0000","last_fullsized":"2024-09-17T03:14:57.341150+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:06:37.330728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.338642+0000","last_change":"2024-09-17T03:14:56.336542+0000","last_active":"2024-09-17T03:14:56.338642+0000","last_peered":"2024-09-17T03:14:56.338642+0000","last_clean":"2024-09-17T03:14:56.338642+0000","last_became_active":"2024-09-17T03:14:56.336216+0000","last_became_peered":"2024-09-17T03:14:56.336216+0000","last_unstale":"2024-09-17T03:14:56.338642+0000","last_undegraded":"2024-09-17T03:14:56.338642+0000","last_fullsized":"2024-09-17T03:14:56.338642+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:10.905183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.355337+0000","last_change":"2024-09-17T03:14:55.335726+0000","last_active":"2024-09-17T03:14:56.355337+0000","last_peered":"2024-09-17T03:14:56.355337+0000","last_clean":"2024-09-17T03:14:56.355337+0000","last_became_active":"2024-09-17T03:14:55.335268+0000","last_became_peered":"2024-09-17T03:14:55.335268+0000","last_unstale":"2024-09-17T03:14:56.355337+0000","last_undegraded":"2024-09-17T03:14:56.355337+0000","last_fullsized":"2024-09-17T03:14:56.355337+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:13.660225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344275+0000","last_change":"2024-09-17T03:14:56.336628+0000","last_active":"2024-09-17T03:14:57.344275+0000","last_peered":"2024-09-17T03:14:57.344275+0000","last_clean":"2024-09-17T03:14:57.344275+0000","last_became_active":"2024-09-17T03:14:56.336330+0000","last_became_peered":"2024-09-17T03:14:56.336330+0000","last_unstale":"2024-09-17T03:14:57.344275+0000","last_undegraded":"2024-09-17T03:14:57.344275+0000","last_fullsized":"2024-09-17T03:14:57.344275+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:02:13.758412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351038+0000","last_change":"2024-09-17T03:14:55.324166+0000","last_active":"2024-09-17T03:14:56.351038+0000","last_peered":"2024-09-17T03:14:56.351038+0000","last_clean":"2024-09-17T03:14:56.351038+0000","last_became_active":"2024-09-17T03:14:55.323844+0000","last_became_peered":"2024-09-17T03:14:55.323844+0000","last_unstale":"2024-09-17T03:14:56.351038+0000","last_undegraded":"2024-09-17T03:14:56.351038+0000","last_fullsized":"2024-09-17T03:14:56.351038+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:27:53.850399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341186+0000","last_change":"2024-09-17T03:14:56.337913+0000","last_active":"2024-09-17T03:14:57.341186+0000","last_peered":"2024-09-17T03:14:57.341186+0000","last_clean":"2024-09-17T03:14:57.341186+0000","last_became_active":"2024-09-17T03:14:56.337610+0000","last_became_peered":"2024-09-17T03:14:56.337610+0000","last_unstale":"2024-09-17T03:14:57.341186+0000","last_undegraded":"2024-09-17T03:14:57.341186+0000","last_fullsized":"2024-09-17T03:14:57.341186+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:08.606153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.325094+0000","last_change":"2024-09-17T03:14:55.333099+0000","last_active":"2024-09-17T03:14:56.325094+0000","last_peered":"2024-09-17T03:14:56.325094+0000","last_clean":"2024-09-17T03:14:56.325094+0000","last_became_active":"2024-09-17T03:14:55.332738+0000","last_became_peered":"2024-09-17T03:14:55.332738+0000","last_unstale":"2024-09-17T03:14:56.325094+0000","last_undegraded":"2024-09-17T03:14:56.325094+0000","last_fullsized":"2024-09-17T03:14:56.325094+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:50.915622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.344808+0000","last_change":"2024-09-17T03:14:56.340119+0000","last_active":"2024-09-17T03:14:56.344808+0000","last_peered":"2024-09-17T03:14:56.344808+0000","last_clean":"2024-09-17T03:14:56.344808+0000","last_became_active":"2024-09-17T03:14:56.339795+0000","last_became_peered":"2024-09-17T03:14:56.339795+0000","last_unstale":"2024-09-17T03:14:56.344808+0000","last_undegraded":"2024-09-17T03:14:56.344808+0000","last_fullsized":"2024-09-17T03:14:56.344808+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:42:04.002930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344218+0000","last_change":"2024-09-17T03:14:55.327065+0000","last_active":"2024-09-17T03:14:57.344218+0000","last_peered":"2024-09-17T03:14:57.344218+0000","last_clean":"2024-09-17T03:14:57.344218+0000","last_became_active":"2024-09-17T03:14:55.326774+0000","last_became_peered":"2024-09-17T03:14:55.326774+0000","last_unstale":"2024-09-17T03:14:57.344218+0000","last_undegraded":"2024-09-17T03:14:57.344218+0000","last_fullsized":"2024-09-17T03:14:57.344218+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:37.138244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344009+0000","last_change":"2024-09-17T03:14:56.335950+0000","last_active":"2024-09-17T03:14:57.344009+0000","last_peered":"2024-09-17T03:14:57.344009+0000","last_clean":"2024-09-17T03:14:57.344009+0000","last_became_active":"2024-09-17T03:14:56.335641+0000","last_became_peered":"2024-09-17T03:14:56.335641+0000","last_unstale":"2024-09-17T03:14:57.344009+0000","last_undegraded":"2024-09-17T03:14:57.344009+0000","last_fullsized":"2024-09-17T03:14:57.344009+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:03.609971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.329227+0000","last_change":"2024-09-17T03:14:55.335625+0000","last_active":"2024-09-17T03:14:56.329227+0000","last_peered":"2024-09-17T03:14:56.329227+0000","last_clean":"2024-09-17T03:14:56.329227+0000","last_became_active":"2024-09-17T03:14:55.335317+0000","last_became_peered":"2024-09-17T03:14:55.335317+0000","last_unstale":"2024-09-17T03:14:56.329227+0000","last_undegraded":"2024-09-17T03:14:56.329227+0000","last_fullsized":"2024-09-17T03:14:56.329227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:47.254999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.357401+0000","last_change":"2024-09-17T03:14:56.346898+0000","last_active":"2024-09-17T03:14:56.357401+0000","last_peered":"2024-09-17T03:14:56.357401+0000","last_clean":"2024-09-17T03:14:56.357401+0000","last_became_active":"2024-09-17T03:14:56.346603+0000","last_became_peered":"2024-09-17T03:14:56.346603+0000","last_unstale":"2024-09-17T03:14:56.357401+0000","last_undegraded":"2024-09-17T03:14:56.357401+0000","last_fullsized":"2024-09-17T03:14:56.357401+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:26:13.257738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.328281+0000","last_change":"2024-09-17T03:14:55.328041+0000","last_active":"2024-09-17T03:14:56.328281+0000","last_peered":"2024-09-17T03:14:56.328281+0000","last_clean":"2024-09-17T03:14:56.328281+0000","last_became_active":"2024-09-17T03:14:55.327533+0000","last_became_peered":"2024-09-17T03:14:55.327533+0000","last_unstale":"2024-09-17T03:14:56.328281+0000","last_undegraded":"2024-09-17T03:14:56.328281+0000","last_fullsized":"2024-09-17T03:14:56.328281+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:21.326711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.337735+0000","last_change":"2024-09-17T03:14:56.336599+0000","last_active":"2024-09-17T03:14:56.337735+0000","last_peered":"2024-09-17T03:14:56.337735+0000","last_clean":"2024-09-17T03:14:56.337735+0000","last_became_active":"2024-09-17T03:14:56.336243+0000","last_became_peered":"2024-09-17T03:14:56.336243+0000","last_unstale":"2024-09-17T03:14:56.337735+0000","last_undegraded":"2024-09-17T03:14:56.337735+0000","last_fullsized":"2024-09-17T03:14:56.337735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:06:38.336520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.329616+0000","last_change":"2024-09-17T03:14:55.324984+0000","last_active":"2024-09-17T03:14:56.329616+0000","last_peered":"2024-09-17T03:14:56.329616+0000","last_clean":"2024-09-17T03:14:56.329616+0000","last_became_active":"2024-09-17T03:14:55.324696+0000","last_became_peered":"2024-09-17T03:14:55.324696+0000","last_unstale":"2024-09-17T03:14:56.329616+0000","last_undegraded":"2024-09-17T03:14:56.329616+0000","last_fullsized":"2024-09-17T03:14:56.329616+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:10.371792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341211+0000","last_change":"2024-09-17T03:14:56.338902+0000","last_active":"2024-09-17T03:14:57.341211+0000","last_peered":"2024-09-17T03:14:57.341211+0000","last_clean":"2024-09-17T03:14:57.341211+0000","last_became_active":"2024-09-17T03:14:56.338504+0000","last_became_peered":"2024-09-17T03:14:56.338504+0000","last_unstale":"2024-09-17T03:14:57.341211+0000","last_undegraded":"2024-09-17T03:14:57.341211+0000","last_fullsized":"2024-09-17T03:14:57.341211+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:16.330515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.357201+0000","last_change":"2024-09-17T03:14:55.328618+0000","last_active":"2024-09-17T03:14:56.357201+0000","last_peered":"2024-09-17T03:14:56.357201+0000","last_clean":"2024-09-17T03:14:56.357201+0000","last_became_active":"2024-09-17T03:14:55.328321+0000","last_became_peered":"2024-09-17T03:14:55.328321+0000","last_unstale":"2024-09-17T03:14:56.357201+0000","last_undegraded":"2024-09-17T03:14:56.357201+0000","last_fullsized":"2024-09-17T03:14:56.357201+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:30:39.180289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.354677+0000","last_change":"2024-09-17T03:14:56.341818+0000","last_active":"2024-09-17T03:14:56.354677+0000","last_peered":"2024-09-17T03:14:56.354677+0000","last_clean":"2024-09-17T03:14:56.354677+0000","last_became_active":"2024-09-17T03:14:56.341526+0000","last_became_peered":"2024-09-17T03:14:56.341526+0000","last_unstale":"2024-09-17T03:14:56.354677+0000","last_undegraded":"2024-09-17T03:14:56.354677+0000","last_fullsized":"2024-09-17T03:14:56.354677+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:35:35.033454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344149+0000","last_change":"2024-09-17T03:14:55.325076+0000","last_active":"2024-09-17T03:14:57.344149+0000","last_peered":"2024-09-17T03:14:57.344149+0000","last_clean":"2024-09-17T03:14:57.344149+0000","last_became_active":"2024-09-17T03:14:55.324738+0000","last_became_peered":"2024-09-17T03:14:55.324738+0000","last_unstale":"2024-09-17T03:14:57.344149+0000","last_undegraded":"2024-09-17T03:14:57.344149+0000","last_fullsized":"2024-09-17T03:14:57.344149+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:26:33.226093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341372+0000","last_change":"2024-09-17T03:14:56.348451+0000","last_active":"2024-09-17T03:14:57.341372+0000","last_peered":"2024-09-17T03:14:57.341372+0000","last_clean":"2024-09-17T03:14:57.341372+0000","last_became_active":"2024-09-17T03:14:56.348169+0000","last_became_peered":"2024-09-17T03:14:56.348169+0000","last_unstale":"2024-09-17T03:14:57.341372+0000","last_undegraded":"2024-09-17T03:14:57.341372+0000","last_fullsized":"2024-09-17T03:14:57.341372+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:47.114511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.332977+0000","last_change":"2024-09-17T03:14:55.327511+0000","last_active":"2024-09-17T03:14:56.332977+0000","last_peered":"2024-09-17T03:14:56.332977+0000","last_clean":"2024-09-17T03:14:56.332977+0000","last_became_active":"2024-09-17T03:14:55.327195+0000","last_became_peered":"2024-09-17T03:14:55.327195+0000","last_unstale":"2024-09-17T03:14:56.332977+0000","last_undegraded":"2024-09-17T03:14:56.332977+0000","last_fullsized":"2024-09-17T03:14:56.332977+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:13.660225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.344622+0000","last_change":"2024-09-17T03:14:56.337722+0000","last_active":"2024-09-17T03:14:56.344622+0000","last_peered":"2024-09-17T03:14:56.344622+0000","last_clean":"2024-09-17T03:14:56.344622+0000","last_became_active":"2024-09-17T03:14:56.337353+0000","last_became_peered":"2024-09-17T03:14:56.337353+0000","last_unstale":"2024-09-17T03:14:56.344622+0000","last_undegraded":"2024-09-17T03:14:56.344622+0000","last_fullsized":"2024-09-17T03:14:56.344622+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:34:03.584090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.337538+0000","last_change":"2024-09-17T03:14:55.328438+0000","last_active":"2024-09-17T03:14:56.337538+0000","last_peered":"2024-09-17T03:14:56.337538+0000","last_clean":"2024-09-17T03:14:56.337538+0000","last_became_active":"2024-09-17T03:14:55.328158+0000","last_became_peered":"2024-09-17T03:14:55.328158+0000","last_unstale":"2024-09-17T03:14:56.337538+0000","last_undegraded":"2024-09-17T03:14:56.337538+0000","last_fullsized":"2024-09-17T03:14:56.337538+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:19:28.098924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.331672+0000","last_change":"2024-09-17T03:14:55.330495+0000","last_active":"2024-09-17T03:14:56.331672+0000","last_peered":"2024-09-17T03:14:56.331672+0000","last_clean":"2024-09-17T03:14:56.331672+0000","last_became_active":"2024-09-17T03:14:55.330159+0000","last_became_peered":"2024-09-17T03:14:55.330159+0000","last_unstale":"2024-09-17T03:14:56.331672+0000","last_undegraded":"2024-09-17T03:14:56.331672+0000","last_fullsized":"2024-09-17T03:14:56.331672+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:55.212014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:14:55.307935+0000","last_change":"2024-09-17T03:14:55.307935+0000","last_active":"2024-09-17T03:14:55.307935+0000","last_peered":"2024-09-17T03:14:55.307935+0000","last_clean":"2024-09-17T03:14:55.307935+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:55.307935+0000","last_undegraded":"2024-09-17T03:14:55.307935+0000","last_fullsized":"2024-09-17T03:14:55.307935+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.331656+0000","last_change":"2024-09-17T03:14:55.331670+0000","last_active":"2024-09-17T03:14:56.331656+0000","last_peered":"2024-09-17T03:14:56.331656+0000","last_clean":"2024-09-17T03:14:56.331656+0000","last_became_active":"2024-09-17T03:14:55.331359+0000","last_became_peered":"2024-09-17T03:14:55.331359+0000","last_unstale":"2024-09-17T03:14:56.331656+0000","last_undegraded":"2024-09-17T03:14:56.331656+0000","last_fullsized":"2024-09-17T03:14:56.331656+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:55.212014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.351181+0000","last_change":"2024-09-17T03:14:56.341402+0000","last_active":"2024-09-17T03:14:56.351181+0000","last_peered":"2024-09-17T03:14:56.351181+0000","last_clean":"2024-09-17T03:14:56.351181+0000","last_became_active":"2024-09-17T03:14:56.341095+0000","last_became_peered":"2024-09-17T03:14:56.341095+0000","last_unstale":"2024-09-17T03:14:56.351181+0000","last_undegraded":"2024-09-17T03:14:56.351181+0000","last_fullsized":"2024-09-17T03:14:56.351181+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:33:09.441087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-09-17T03:14:54.312546+0000","last_change":"2024-09-17T03:14:54.312546+0000","last_active":"2024-09-17T03:14:54.302142+0000","last_peered":"2024-09-17T03:14:54.302142+0000","last_clean":"2024-09-17T03:14:54.302142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:14:54.312546+0000","last_undegraded":"2024-09-17T03:14:54.312546+0000","last_fullsized":"2024-09-17T03:14:54.312546+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.343160+0000","last_change":"2024-09-17T03:14:56.337604+0000","last_active":"2024-09-17T03:14:56.343160+0000","last_peered":"2024-09-17T03:14:56.343160+0000","last_clean":"2024-09-17T03:14:56.343160+0000","last_became_active":"2024-09-17T03:14:56.337283+0000","last_became_peered":"2024-09-17T03:14:56.337283+0000","last_unstale":"2024-09-17T03:14:56.343160+0000","last_undegraded":"2024-09-17T03:14:56.343160+0000","last_fullsized":"2024-09-17T03:14:56.343160+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:11:17.535415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.333475+0000","last_change":"2024-09-17T03:14:55.322959+0000","last_active":"2024-09-17T03:14:56.333475+0000","last_peered":"2024-09-17T03:14:56.333475+0000","last_clean":"2024-09-17T03:14:56.333475+0000","last_became_active":"2024-09-17T03:14:55.322475+0000","last_became_peered":"2024-09-17T03:14:55.322475+0000","last_unstale":"2024-09-17T03:14:56.333475+0000","last_undegraded":"2024-09-17T03:14:56.333475+0000","last_fullsized":"2024-09-17T03:14:56.333475+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:57.870235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":127,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-17T03:14:56.328846+0000","last_change":"2024-09-17T03:14:04.862167+0000","last_active":"2024-09-17T03:14:56.328846+0000","last_peered":"2024-09-17T03:14:56.328846+0000","last_clean":"2024-09-17T03:14:56.328846+0000","last_became_active":"2024-09-17T03:14:04.861731+0000","last_became_peered":"2024-09-17T03:14:04.861731+0000","last_unstale":"2024-09-17T03:14:56.328846+0000","last_undegraded":"2024-09-17T03:14:56.328846+0000","last_fullsized":"2024-09-17T03:14:56.328846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_clean_scrub_stamp":"2024-09-17T03:14:03.847721+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:32.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":104,"acting":104,"num_store_stats":7},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509456,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":368640,"data_stored":139029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":376832,"data_stored":140619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509455,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32136,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339704,"statfs":{"total":96636764160,"available":96603856896,"internally_reserved":0,"allocated":962560,"data_stored":729957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27404,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344436,"statfs":{"total":96636764160,"available":96608702464,"internally_reserved":0,"allocated":380928,"data_stored":139783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":368640,"data_stored":139041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509456,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":962560,"data_stored":729445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27392,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":368640,"data_stored":139097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":327680,"data_stored":120140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:15:03.770 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:15:03.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.898+0000 7fe3440da640 1 -- 172.21.15.87:0/1324219793 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe328003a50 msgr2=0x7fe328003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:03.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.898+0000 7fe3440da640 1 --2- 172.21.15.87:0/1324219793 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe328003a50 0x7fe328003ec0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe3300016c0 tx=0x7fe328005830 comp rx=0 tx=0).stop 2024-09-17T03:15:03.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.898+0000 7fe3440da640 1 -- 172.21.15.87:0/1324219793 shutdown_connections 2024-09-17T03:15:03.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.898+0000 7fe3440da640 1 --2- 172.21.15.87:0/1324219793 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe328003a50 0x7fe328003ec0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:03.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.898+0000 7fe3440da640 1 -- 172.21.15.87:0/1324219793 >> 172.21.15.87:0/1324219793 conn(0x7fe33c0d1110 msgr2=0x7fe33c0d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:03.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.898+0000 7fe3440da640 1 -- 172.21.15.87:0/1324219793 shutdown_connections 2024-09-17T03:15:03.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.898+0000 7fe3440da640 1 -- 172.21.15.87:0/1324219793 wait complete. 2024-09-17T03:15:03.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.900+0000 7fe3440da640 1 Processor -- start 2024-09-17T03:15:03.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.900+0000 7fe3440da640 1 -- start start 2024-09-17T03:15:03.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.901+0000 7fe3440da640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe328003a50 0x7fe33c0ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:03.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.901+0000 7fe3440da640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe33c1553d0 0x7fe33c1578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:03.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.901+0000 7fe3440da640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe33c155920 0x7fe33c157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:03.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.901+0000 7fe3440da640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c15f1f0 con 0x7fe328003a50 2024-09-17T03:15:03.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.901+0000 7fe3440da640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c15f070 con 0x7fe33c1553d0 2024-09-17T03:15:03.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.901+0000 7fe3440da640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe33c15f370 con 0x7fe33c155920 2024-09-17T03:15:03.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.901+0000 7fe341e76640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe328003a50 0x7fe33c0ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:03.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.901+0000 7fe341e76640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe328003a50 0x7fe33c0ca0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40056/0 (socket says 172.21.15.87:40056) 2024-09-17T03:15:03.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.901+0000 7fe341e76640 1 -- 172.21.15.87:0/1649977605 learned_addr learned my addr 172.21.15.87:0/1649977605 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:03.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.901+0000 7fe342677640 1 --2- 172.21.15.87:0/1649977605 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe33c155920 0x7fe33c157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:03.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.901+0000 7fe341675640 1 --2- 172.21.15.87:0/1649977605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe33c1553d0 0x7fe33c1578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:03.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.902+0000 7fe341e76640 1 -- 172.21.15.87:0/1649977605 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe33c155920 msgr2=0x7fe33c157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:03.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.902+0000 7fe341e76640 1 --2- 172.21.15.87:0/1649977605 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe33c155920 0x7fe33c157e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:03.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.902+0000 7fe341e76640 1 -- 172.21.15.87:0/1649977605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe33c1553d0 msgr2=0x7fe33c1578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:03.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.902+0000 7fe341e76640 1 --2- 172.21.15.87:0/1649977605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe33c1553d0 0x7fe33c1578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:03.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.902+0000 7fe341e76640 1 -- 172.21.15.87:0/1649977605 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe33c0ca630 con 0x7fe328003a50 2024-09-17T03:15:03.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.902+0000 7fe341e76640 1 --2- 172.21.15.87:0/1649977605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe328003a50 0x7fe33c0ca0f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe3300017e0 tx=0x7fe3300063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:03.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.902+0000 7fe32e7fc640 1 -- 172.21.15.87:0/1649977605 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330042070 con 0x7fe328003a50 2024-09-17T03:15:03.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.902+0000 7fe32e7fc640 1 -- 172.21.15.87:0/1649977605 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe330039070 con 0x7fe328003a50 2024-09-17T03:15:03.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.902+0000 7fe3440da640 1 -- 172.21.15.87:0/1649977605 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe33c1f6b10 con 0x7fe328003a50 2024-09-17T03:15:03.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.902+0000 7fe32e7fc640 1 -- 172.21.15.87:0/1649977605 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe330003d90 con 0x7fe328003a50 2024-09-17T03:15:03.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.902+0000 7fe3440da640 1 -- 172.21.15.87:0/1649977605 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe33c1f70d0 con 0x7fe328003a50 2024-09-17T03:15:03.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.903+0000 7fe3440da640 1 -- 172.21.15.87:0/1649977605 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe33c0b0180 con 0x7fe328003a50 2024-09-17T03:15:03.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.904+0000 7fe32e7fc640 1 -- 172.21.15.87:0/1649977605 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe33003b070 con 0x7fe328003a50 2024-09-17T03:15:03.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.904+0000 7fe32e7fc640 1 --2- 172.21.15.87:0/1649977605 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe320078bc0 0x7fe32007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:03.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.904+0000 7fe32e7fc640 1 -- 172.21.15.87:0/1649977605 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fe3300ba190 con 0x7fe328003a50 2024-09-17T03:15:03.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.904+0000 7fe341675640 1 --2- 172.21.15.87:0/1649977605 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe320078bc0 0x7fe32007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:03.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.905+0000 7fe341675640 1 --2- 172.21.15.87:0/1649977605 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe320078bc0 0x7fe32007b0d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe33c06dbb0 tx=0x7fe338005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:03.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:03.909+0000 7fe32e7fc640 1 -- 172.21.15.87:0/1649977605 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe330082040 con 0x7fe328003a50 2024-09-17T03:15:04.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.081+0000 7fe3440da640 1 -- 172.21.15.87:0/1649977605 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe33c0d0290 con 0x7fe320078bc0 2024-09-17T03:15:04.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.086+0000 7fe32e7fc640 1 -- 172.21.15.87:0/1649977605 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251892 (secure 0 0 0) 0x7fe33805e810 con 0x7fe320078bc0 2024-09-17T03:15:04.087 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:04.088 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-09-17T03:15:04.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.089+0000 7fe3440da640 1 -- 172.21.15.87:0/1649977605 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe320078bc0 msgr2=0x7fe32007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:04.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.089+0000 7fe3440da640 1 --2- 172.21.15.87:0/1649977605 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe320078bc0 0x7fe32007b0d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe33c06dbb0 tx=0x7fe338005f90 comp rx=0 tx=0).stop 2024-09-17T03:15:04.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.089+0000 7fe3440da640 1 -- 172.21.15.87:0/1649977605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe328003a50 msgr2=0x7fe33c0ca0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:04.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.089+0000 7fe3440da640 1 --2- 172.21.15.87:0/1649977605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe328003a50 0x7fe33c0ca0f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe3300017e0 tx=0x7fe3300063a0 comp rx=0 tx=0).stop 2024-09-17T03:15:04.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.089+0000 7fe3440da640 1 -- 172.21.15.87:0/1649977605 shutdown_connections 2024-09-17T03:15:04.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.089+0000 7fe3440da640 1 --2- 172.21.15.87:0/1649977605 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe320078bc0 0x7fe32007b0d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:04.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.089+0000 7fe3440da640 1 --2- 172.21.15.87:0/1649977605 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe33c155920 0x7fe33c157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:04.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.089+0000 7fe3440da640 1 --2- 172.21.15.87:0/1649977605 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe33c1553d0 0x7fe33c1578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:04.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.089+0000 7fe3440da640 1 --2- 172.21.15.87:0/1649977605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe328003a50 0x7fe33c0ca0f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:04.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.089+0000 7fe3440da640 1 -- 172.21.15.87:0/1649977605 >> 172.21.15.87:0/1649977605 conn(0x7fe33c0d1110 msgr2=0x7fe33c06ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:04.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.089+0000 7fe3440da640 1 -- 172.21.15.87:0/1649977605 shutdown_connections 2024-09-17T03:15:04.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.089+0000 7fe3440da640 1 -- 172.21.15.87:0/1649977605 wait complete. 2024-09-17T03:15:04.099 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2024-09-17T03:15:03.428324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224560,"kb_used_data":4168,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754750160,"statfs":{"total":773094113280,"available":772864163840,"internally_reserved":0,"allocated":4268032,"data_stored":2348012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002073"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.582789+0000","last_change":"2024-09-17T03:14:56.344756+0000","last_active":"2024-09-17T03:14:58.582789+0000","last_peered":"2024-09-17T03:14:58.582789+0000","last_clean":"2024-09-17T03:14:58.582789+0000","last_became_active":"2024-09-17T03:14:56.343497+0000","last_became_peered":"2024-09-17T03:14:56.343497+0000","last_unstale":"2024-09-17T03:14:58.582789+0000","last_undegraded":"2024-09-17T03:14:58.582789+0000","last_fullsized":"2024-09-17T03:14:58.582789+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:40.283002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344120+0000","last_change":"2024-09-17T03:14:56.348503+0000","last_active":"2024-09-17T03:14:57.344120+0000","last_peered":"2024-09-17T03:14:57.344120+0000","last_clean":"2024-09-17T03:14:57.344120+0000","last_became_active":"2024-09-17T03:14:56.348172+0000","last_became_peered":"2024-09-17T03:14:56.348172+0000","last_unstale":"2024-09-17T03:14:57.344120+0000","last_undegraded":"2024-09-17T03:14:57.344120+0000","last_fullsized":"2024-09-17T03:14:57.344120+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:52:09.125634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583902+0000","last_change":"2024-09-17T03:14:56.347013+0000","last_active":"2024-09-17T03:14:58.583902+0000","last_peered":"2024-09-17T03:14:58.583902+0000","last_clean":"2024-09-17T03:14:58.583902+0000","last_became_active":"2024-09-17T03:14:56.346718+0000","last_became_peered":"2024-09-17T03:14:56.346718+0000","last_unstale":"2024-09-17T03:14:58.583902+0000","last_undegraded":"2024-09-17T03:14:58.583902+0000","last_fullsized":"2024-09-17T03:14:58.583902+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:08.606153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.345956+0000","last_change":"2024-09-17T03:14:56.352444+0000","last_active":"2024-09-17T03:14:57.345956+0000","last_peered":"2024-09-17T03:14:57.345956+0000","last_clean":"2024-09-17T03:14:57.345956+0000","last_became_active":"2024-09-17T03:14:56.351998+0000","last_became_peered":"2024-09-17T03:14:56.351998+0000","last_unstale":"2024-09-17T03:14:57.345956+0000","last_undegraded":"2024-09-17T03:14:57.345956+0000","last_fullsized":"2024-09-17T03:14:57.345956+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:32:01.437684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.343763+0000","last_change":"2024-09-17T03:14:56.344528+0000","last_active":"2024-09-17T03:14:57.343763+0000","last_peered":"2024-09-17T03:14:57.343763+0000","last_clean":"2024-09-17T03:14:57.343763+0000","last_became_active":"2024-09-17T03:14:56.344226+0000","last_became_peered":"2024-09-17T03:14:56.344226+0000","last_unstale":"2024-09-17T03:14:57.343763+0000","last_undegraded":"2024-09-17T03:14:57.343763+0000","last_fullsized":"2024-09-17T03:14:57.343763+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:09:48.039023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.340976+0000","last_change":"2024-09-17T03:14:56.342382+0000","last_active":"2024-09-17T03:14:57.340976+0000","last_peered":"2024-09-17T03:14:57.340976+0000","last_clean":"2024-09-17T03:14:57.340976+0000","last_became_active":"2024-09-17T03:14:56.341980+0000","last_became_peered":"2024-09-17T03:14:56.341980+0000","last_unstale":"2024-09-17T03:14:57.340976+0000","last_undegraded":"2024-09-17T03:14:57.340976+0000","last_fullsized":"2024-09-17T03:14:57.340976+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:11.714679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.582585+0000","last_change":"2024-09-17T03:14:56.343019+0000","last_active":"2024-09-17T03:14:58.582585+0000","last_peered":"2024-09-17T03:14:58.582585+0000","last_clean":"2024-09-17T03:14:58.582585+0000","last_became_active":"2024-09-17T03:14:56.342705+0000","last_became_peered":"2024-09-17T03:14:56.342705+0000","last_unstale":"2024-09-17T03:14:58.582585+0000","last_undegraded":"2024-09-17T03:14:58.582585+0000","last_fullsized":"2024-09-17T03:14:58.582585+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:16:28.826808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.582683+0000","last_change":"2024-09-17T03:14:56.346692+0000","last_active":"2024-09-17T03:14:58.582683+0000","last_peered":"2024-09-17T03:14:58.582683+0000","last_clean":"2024-09-17T03:14:58.582683+0000","last_became_active":"2024-09-17T03:14:56.346397+0000","last_became_peered":"2024-09-17T03:14:56.346397+0000","last_unstale":"2024-09-17T03:14:58.582683+0000","last_undegraded":"2024-09-17T03:14:58.582683+0000","last_fullsized":"2024-09-17T03:14:58.582683+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:10:26.148335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.584166+0000","last_change":"2024-09-17T03:14:56.339990+0000","last_active":"2024-09-17T03:14:58.584166+0000","last_peered":"2024-09-17T03:14:58.584166+0000","last_clean":"2024-09-17T03:14:58.584166+0000","last_became_active":"2024-09-17T03:14:56.339281+0000","last_became_peered":"2024-09-17T03:14:56.339281+0000","last_unstale":"2024-09-17T03:14:58.584166+0000","last_undegraded":"2024-09-17T03:14:58.584166+0000","last_fullsized":"2024-09-17T03:14:58.584166+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:16.330515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.345543+0000","last_change":"2024-09-17T03:14:56.346427+0000","last_active":"2024-09-17T03:14:57.345543+0000","last_peered":"2024-09-17T03:14:57.345543+0000","last_clean":"2024-09-17T03:14:57.345543+0000","last_became_active":"2024-09-17T03:14:56.346086+0000","last_became_peered":"2024-09-17T03:14:56.346086+0000","last_unstale":"2024-09-17T03:14:57.345543+0000","last_undegraded":"2024-09-17T03:14:57.345543+0000","last_fullsized":"2024-09-17T03:14:57.345543+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:25:58.344679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.196778+0000","last_change":"2024-09-17T03:14:56.342266+0000","last_active":"2024-09-17T03:14:58.196778+0000","last_peered":"2024-09-17T03:14:58.196778+0000","last_clean":"2024-09-17T03:14:58.196778+0000","last_became_active":"2024-09-17T03:14:56.341978+0000","last_became_peered":"2024-09-17T03:14:56.341978+0000","last_unstale":"2024-09-17T03:14:58.196778+0000","last_undegraded":"2024-09-17T03:14:58.196778+0000","last_fullsized":"2024-09-17T03:14:58.196778+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:14.368939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583647+0000","last_change":"2024-09-17T03:14:56.342630+0000","last_active":"2024-09-17T03:14:58.583647+0000","last_peered":"2024-09-17T03:14:58.583647+0000","last_clean":"2024-09-17T03:14:58.583647+0000","last_became_active":"2024-09-17T03:14:56.342175+0000","last_became_peered":"2024-09-17T03:14:56.342175+0000","last_unstale":"2024-09-17T03:14:58.583647+0000","last_undegraded":"2024-09-17T03:14:58.583647+0000","last_fullsized":"2024-09-17T03:14:58.583647+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:31:11.714679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583219+0000","last_change":"2024-09-17T03:14:56.351892+0000","last_active":"2024-09-17T03:14:58.583219+0000","last_peered":"2024-09-17T03:14:58.583219+0000","last_clean":"2024-09-17T03:14:58.583219+0000","last_became_active":"2024-09-17T03:14:56.351149+0000","last_became_peered":"2024-09-17T03:14:56.351149+0000","last_unstale":"2024-09-17T03:14:58.583219+0000","last_undegraded":"2024-09-17T03:14:58.583219+0000","last_fullsized":"2024-09-17T03:14:58.583219+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:27.164191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.345293+0000","last_change":"2024-09-17T03:14:56.336877+0000","last_active":"2024-09-17T03:14:57.345293+0000","last_peered":"2024-09-17T03:14:57.345293+0000","last_clean":"2024-09-17T03:14:57.345293+0000","last_became_active":"2024-09-17T03:14:56.336549+0000","last_became_peered":"2024-09-17T03:14:56.336549+0000","last_unstale":"2024-09-17T03:14:57.345293+0000","last_undegraded":"2024-09-17T03:14:57.345293+0000","last_fullsized":"2024-09-17T03:14:57.345293+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:13:20.048401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583302+0000","last_change":"2024-09-17T03:14:56.349422+0000","last_active":"2024-09-17T03:14:58.583302+0000","last_peered":"2024-09-17T03:14:58.583302+0000","last_clean":"2024-09-17T03:14:58.583302+0000","last_became_active":"2024-09-17T03:14:56.349132+0000","last_became_peered":"2024-09-17T03:14:56.349132+0000","last_unstale":"2024-09-17T03:14:58.583302+0000","last_undegraded":"2024-09-17T03:14:58.583302+0000","last_fullsized":"2024-09-17T03:14:58.583302+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:11.377584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583088+0000","last_change":"2024-09-17T03:14:56.337002+0000","last_active":"2024-09-17T03:14:58.583088+0000","last_peered":"2024-09-17T03:14:58.583088+0000","last_clean":"2024-09-17T03:14:58.583088+0000","last_became_active":"2024-09-17T03:14:56.336649+0000","last_became_peered":"2024-09-17T03:14:56.336649+0000","last_unstale":"2024-09-17T03:14:58.583088+0000","last_undegraded":"2024-09-17T03:14:58.583088+0000","last_fullsized":"2024-09-17T03:14:58.583088+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:42.736899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.196688+0000","last_change":"2024-09-17T03:14:56.346405+0000","last_active":"2024-09-17T03:14:58.196688+0000","last_peered":"2024-09-17T03:14:58.196688+0000","last_clean":"2024-09-17T03:14:58.196688+0000","last_became_active":"2024-09-17T03:14:56.346037+0000","last_became_peered":"2024-09-17T03:14:56.346037+0000","last_unstale":"2024-09-17T03:14:58.196688+0000","last_undegraded":"2024-09-17T03:14:58.196688+0000","last_fullsized":"2024-09-17T03:14:58.196688+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:57:56.870880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341208+0000","last_change":"2024-09-17T03:14:56.345147+0000","last_active":"2024-09-17T03:14:57.341208+0000","last_peered":"2024-09-17T03:14:57.341208+0000","last_clean":"2024-09-17T03:14:57.341208+0000","last_became_active":"2024-09-17T03:14:56.344844+0000","last_became_peered":"2024-09-17T03:14:56.344844+0000","last_unstale":"2024-09-17T03:14:57.341208+0000","last_undegraded":"2024-09-17T03:14:57.341208+0000","last_fullsized":"2024-09-17T03:14:57.341208+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:30:40.186081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341134+0000","last_change":"2024-09-17T03:14:56.339650+0000","last_active":"2024-09-17T03:14:57.341134+0000","last_peered":"2024-09-17T03:14:57.341134+0000","last_clean":"2024-09-17T03:14:57.341134+0000","last_became_active":"2024-09-17T03:14:56.338719+0000","last_became_peered":"2024-09-17T03:14:56.338719+0000","last_unstale":"2024-09-17T03:14:57.341134+0000","last_undegraded":"2024-09-17T03:14:57.341134+0000","last_fullsized":"2024-09-17T03:14:57.341134+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:51:33.167537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583459+0000","last_change":"2024-09-17T03:14:56.341270+0000","last_active":"2024-09-17T03:14:58.583459+0000","last_peered":"2024-09-17T03:14:58.583459+0000","last_clean":"2024-09-17T03:14:58.583459+0000","last_became_active":"2024-09-17T03:14:56.340876+0000","last_became_peered":"2024-09-17T03:14:56.340876+0000","last_unstale":"2024-09-17T03:14:58.583459+0000","last_undegraded":"2024-09-17T03:14:58.583459+0000","last_fullsized":"2024-09-17T03:14:58.583459+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:32:50.930343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.582917+0000","last_change":"2024-09-17T03:14:56.344410+0000","last_active":"2024-09-17T03:14:58.582917+0000","last_peered":"2024-09-17T03:14:58.582917+0000","last_clean":"2024-09-17T03:14:58.582917+0000","last_became_active":"2024-09-17T03:14:56.344151+0000","last_became_peered":"2024-09-17T03:14:56.344151+0000","last_unstale":"2024-09-17T03:14:58.582917+0000","last_undegraded":"2024-09-17T03:14:58.582917+0000","last_fullsized":"2024-09-17T03:14:58.582917+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:58:48.186752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.584037+0000","last_change":"2024-09-17T03:14:56.354343+0000","last_active":"2024-09-17T03:14:58.584037+0000","last_peered":"2024-09-17T03:14:58.584037+0000","last_clean":"2024-09-17T03:14:58.584037+0000","last_became_active":"2024-09-17T03:14:56.354020+0000","last_became_peered":"2024-09-17T03:14:56.354020+0000","last_unstale":"2024-09-17T03:14:58.584037+0000","last_undegraded":"2024-09-17T03:14:58.584037+0000","last_fullsized":"2024-09-17T03:14:58.584037+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:58.876027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.582998+0000","last_change":"2024-09-17T03:14:56.340466+0000","last_active":"2024-09-17T03:14:58.582998+0000","last_peered":"2024-09-17T03:14:58.582998+0000","last_clean":"2024-09-17T03:14:58.582998+0000","last_became_active":"2024-09-17T03:14:56.340135+0000","last_became_peered":"2024-09-17T03:14:56.340135+0000","last_unstale":"2024-09-17T03:14:58.582998+0000","last_undegraded":"2024-09-17T03:14:58.582998+0000","last_fullsized":"2024-09-17T03:14:58.582998+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:47:04.426421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.582940+0000","last_change":"2024-09-17T03:14:56.353500+0000","last_active":"2024-09-17T03:14:58.582940+0000","last_peered":"2024-09-17T03:14:58.582940+0000","last_clean":"2024-09-17T03:14:58.582940+0000","last_became_active":"2024-09-17T03:14:56.353211+0000","last_became_peered":"2024-09-17T03:14:56.353211+0000","last_unstale":"2024-09-17T03:14:58.582940+0000","last_undegraded":"2024-09-17T03:14:58.582940+0000","last_fullsized":"2024-09-17T03:14:58.582940+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:58:48.186752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.345780+0000","last_change":"2024-09-17T03:14:56.346686+0000","last_active":"2024-09-17T03:14:57.345780+0000","last_peered":"2024-09-17T03:14:57.345780+0000","last_clean":"2024-09-17T03:14:57.345780+0000","last_became_active":"2024-09-17T03:14:56.345942+0000","last_became_peered":"2024-09-17T03:14:56.345942+0000","last_unstale":"2024-09-17T03:14:57.345780+0000","last_undegraded":"2024-09-17T03:14:57.345780+0000","last_fullsized":"2024-09-17T03:14:57.345780+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:39:56.746227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.345919+0000","last_change":"2024-09-17T03:14:56.342458+0000","last_active":"2024-09-17T03:14:57.345919+0000","last_peered":"2024-09-17T03:14:57.345919+0000","last_clean":"2024-09-17T03:14:57.345919+0000","last_became_active":"2024-09-17T03:14:56.342110+0000","last_became_peered":"2024-09-17T03:14:56.342110+0000","last_unstale":"2024-09-17T03:14:57.345919+0000","last_undegraded":"2024-09-17T03:14:57.345919+0000","last_fullsized":"2024-09-17T03:14:57.345919+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:28:21.114427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.343839+0000","last_change":"2024-09-17T03:14:56.329704+0000","last_active":"2024-09-17T03:14:57.343839+0000","last_peered":"2024-09-17T03:14:57.343839+0000","last_clean":"2024-09-17T03:14:57.343839+0000","last_became_active":"2024-09-17T03:14:56.329354+0000","last_became_peered":"2024-09-17T03:14:56.329354+0000","last_unstale":"2024-09-17T03:14:57.343839+0000","last_undegraded":"2024-09-17T03:14:57.343839+0000","last_fullsized":"2024-09-17T03:14:57.343839+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:59:50.923956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583641+0000","last_change":"2024-09-17T03:14:56.348309+0000","last_active":"2024-09-17T03:14:58.583641+0000","last_peered":"2024-09-17T03:14:58.583641+0000","last_clean":"2024-09-17T03:14:58.583641+0000","last_became_active":"2024-09-17T03:14:56.347990+0000","last_became_peered":"2024-09-17T03:14:56.347990+0000","last_unstale":"2024-09-17T03:14:58.583641+0000","last_undegraded":"2024-09-17T03:14:58.583641+0000","last_fullsized":"2024-09-17T03:14:58.583641+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:46.034257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583098+0000","last_change":"2024-09-17T03:14:56.342697+0000","last_active":"2024-09-17T03:14:58.583098+0000","last_peered":"2024-09-17T03:14:58.583098+0000","last_clean":"2024-09-17T03:14:58.583098+0000","last_became_active":"2024-09-17T03:14:56.342375+0000","last_became_peered":"2024-09-17T03:14:56.342375+0000","last_unstale":"2024-09-17T03:14:58.583098+0000","last_undegraded":"2024-09-17T03:14:58.583098+0000","last_fullsized":"2024-09-17T03:14:58.583098+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:25:02.027713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583327+0000","last_change":"2024-09-17T03:14:56.346711+0000","last_active":"2024-09-17T03:14:58.583327+0000","last_peered":"2024-09-17T03:14:58.583327+0000","last_clean":"2024-09-17T03:14:58.583327+0000","last_became_active":"2024-09-17T03:14:56.346399+0000","last_became_peered":"2024-09-17T03:14:56.346399+0000","last_unstale":"2024-09-17T03:14:58.583327+0000","last_undegraded":"2024-09-17T03:14:58.583327+0000","last_fullsized":"2024-09-17T03:14:58.583327+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:16:28.826808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583227+0000","last_change":"2024-09-17T03:14:56.348432+0000","last_active":"2024-09-17T03:14:58.583227+0000","last_peered":"2024-09-17T03:14:58.583227+0000","last_clean":"2024-09-17T03:14:58.583227+0000","last_became_active":"2024-09-17T03:14:56.348048+0000","last_became_peered":"2024-09-17T03:14:56.348048+0000","last_unstale":"2024-09-17T03:14:58.583227+0000","last_undegraded":"2024-09-17T03:14:58.583227+0000","last_fullsized":"2024-09-17T03:14:58.583227+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:25:02.027713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.196405+0000","last_change":"2024-09-17T03:14:56.346523+0000","last_active":"2024-09-17T03:14:58.196405+0000","last_peered":"2024-09-17T03:14:58.196405+0000","last_clean":"2024-09-17T03:14:58.196405+0000","last_became_active":"2024-09-17T03:14:56.346207+0000","last_became_peered":"2024-09-17T03:14:56.346207+0000","last_unstale":"2024-09-17T03:14:58.196405+0000","last_undegraded":"2024-09-17T03:14:58.196405+0000","last_fullsized":"2024-09-17T03:14:58.196405+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:53:14.646681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583615+0000","last_change":"2024-09-17T03:14:56.341373+0000","last_active":"2024-09-17T03:14:58.583615+0000","last_peered":"2024-09-17T03:14:58.583615+0000","last_clean":"2024-09-17T03:14:58.583615+0000","last_became_active":"2024-09-17T03:14:56.341066+0000","last_became_peered":"2024-09-17T03:14:56.341066+0000","last_unstale":"2024-09-17T03:14:58.583615+0000","last_undegraded":"2024-09-17T03:14:58.583615+0000","last_fullsized":"2024-09-17T03:14:58.583615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:33:59.641065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.196912+0000","last_change":"2024-09-17T03:14:55.321527+0000","last_active":"2024-09-17T03:14:58.196912+0000","last_peered":"2024-09-17T03:14:58.196912+0000","last_clean":"2024-09-17T03:14:58.196912+0000","last_became_active":"2024-09-17T03:14:55.320714+0000","last_became_peered":"2024-09-17T03:14:55.320714+0000","last_unstale":"2024-09-17T03:14:58.196912+0000","last_undegraded":"2024-09-17T03:14:58.196912+0000","last_fullsized":"2024-09-17T03:14:58.196912+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:16:27.821016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583037+0000","last_change":"2024-09-17T03:14:56.344463+0000","last_active":"2024-09-17T03:14:58.583037+0000","last_peered":"2024-09-17T03:14:58.583037+0000","last_clean":"2024-09-17T03:14:58.583037+0000","last_became_active":"2024-09-17T03:14:56.344125+0000","last_became_peered":"2024-09-17T03:14:56.344125+0000","last_unstale":"2024-09-17T03:14:58.583037+0000","last_undegraded":"2024-09-17T03:14:58.583037+0000","last_fullsized":"2024-09-17T03:14:58.583037+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:51.431874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.345628+0000","last_change":"2024-09-17T03:14:55.331656+0000","last_active":"2024-09-17T03:14:57.345628+0000","last_peered":"2024-09-17T03:14:57.345628+0000","last_clean":"2024-09-17T03:14:57.345628+0000","last_became_active":"2024-09-17T03:14:55.330881+0000","last_became_peered":"2024-09-17T03:14:55.330881+0000","last_unstale":"2024-09-17T03:14:57.345628+0000","last_undegraded":"2024-09-17T03:14:57.345628+0000","last_fullsized":"2024-09-17T03:14:57.345628+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:46:01.721710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583865+0000","last_change":"2024-09-17T03:14:56.354666+0000","last_active":"2024-09-17T03:14:58.583865+0000","last_peered":"2024-09-17T03:14:58.583865+0000","last_clean":"2024-09-17T03:14:58.583865+0000","last_became_active":"2024-09-17T03:14:56.354360+0000","last_became_peered":"2024-09-17T03:14:56.354360+0000","last_unstale":"2024-09-17T03:14:58.583865+0000","last_undegraded":"2024-09-17T03:14:58.583865+0000","last_fullsized":"2024-09-17T03:14:58.583865+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:27:54.856191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341309+0000","last_change":"2024-09-17T03:14:55.332084+0000","last_active":"2024-09-17T03:14:57.341309+0000","last_peered":"2024-09-17T03:14:57.341309+0000","last_clean":"2024-09-17T03:14:57.341309+0000","last_became_active":"2024-09-17T03:14:55.331735+0000","last_became_peered":"2024-09-17T03:14:55.331735+0000","last_unstale":"2024-09-17T03:14:57.341309+0000","last_undegraded":"2024-09-17T03:14:57.341309+0000","last_fullsized":"2024-09-17T03:14:57.341309+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:07:23.002081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.345692+0000","last_change":"2024-09-17T03:14:56.341428+0000","last_active":"2024-09-17T03:14:57.345692+0000","last_peered":"2024-09-17T03:14:57.345692+0000","last_clean":"2024-09-17T03:14:57.345692+0000","last_became_active":"2024-09-17T03:14:56.341148+0000","last_became_peered":"2024-09-17T03:14:56.341148+0000","last_unstale":"2024-09-17T03:14:57.345692+0000","last_undegraded":"2024-09-17T03:14:57.345692+0000","last_fullsized":"2024-09-17T03:14:57.345692+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:35:17.805329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.346051+0000","last_change":"2024-09-17T03:14:55.329694+0000","last_active":"2024-09-17T03:14:57.346051+0000","last_peered":"2024-09-17T03:14:57.346051+0000","last_clean":"2024-09-17T03:14:57.346051+0000","last_became_active":"2024-09-17T03:14:55.329370+0000","last_became_peered":"2024-09-17T03:14:55.329370+0000","last_unstale":"2024-09-17T03:14:57.346051+0000","last_undegraded":"2024-09-17T03:14:57.346051+0000","last_fullsized":"2024-09-17T03:14:57.346051+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:56:44.440837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.582539+0000","last_change":"2024-09-17T03:14:56.345924+0000","last_active":"2024-09-17T03:14:58.582539+0000","last_peered":"2024-09-17T03:14:58.582539+0000","last_clean":"2024-09-17T03:14:58.582539+0000","last_became_active":"2024-09-17T03:14:56.345340+0000","last_became_peered":"2024-09-17T03:14:56.345340+0000","last_unstale":"2024-09-17T03:14:58.582539+0000","last_undegraded":"2024-09-17T03:14:58.582539+0000","last_fullsized":"2024-09-17T03:14:58.582539+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:16:18.733102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583136+0000","last_change":"2024-09-17T03:14:55.334211+0000","last_active":"2024-09-17T03:14:58.583136+0000","last_peered":"2024-09-17T03:14:58.583136+0000","last_clean":"2024-09-17T03:14:58.583136+0000","last_became_active":"2024-09-17T03:14:55.333714+0000","last_became_peered":"2024-09-17T03:14:55.333714+0000","last_unstale":"2024-09-17T03:14:58.583136+0000","last_undegraded":"2024-09-17T03:14:58.583136+0000","last_fullsized":"2024-09-17T03:14:58.583136+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:30:39.180289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.582665+0000","last_change":"2024-09-17T03:14:56.348919+0000","last_active":"2024-09-17T03:14:58.582665+0000","last_peered":"2024-09-17T03:14:58.582665+0000","last_clean":"2024-09-17T03:14:58.582665+0000","last_became_active":"2024-09-17T03:14:56.348235+0000","last_became_peered":"2024-09-17T03:14:56.348235+0000","last_unstale":"2024-09-17T03:14:58.582665+0000","last_undegraded":"2024-09-17T03:14:58.582665+0000","last_fullsized":"2024-09-17T03:14:58.582665+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:23:42.286024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583696+0000","last_change":"2024-09-17T03:14:55.335181+0000","last_active":"2024-09-17T03:14:58.583696+0000","last_peered":"2024-09-17T03:14:58.583696+0000","last_clean":"2024-09-17T03:14:58.583696+0000","last_became_active":"2024-09-17T03:14:55.334860+0000","last_became_peered":"2024-09-17T03:14:55.334860+0000","last_unstale":"2024-09-17T03:14:58.583696+0000","last_undegraded":"2024-09-17T03:14:58.583696+0000","last_fullsized":"2024-09-17T03:14:58.583696+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:34:41.387404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.582562+0000","last_change":"2024-09-17T03:14:56.338147+0000","last_active":"2024-09-17T03:14:58.582562+0000","last_peered":"2024-09-17T03:14:58.582562+0000","last_clean":"2024-09-17T03:14:58.582562+0000","last_became_active":"2024-09-17T03:14:56.337801+0000","last_became_peered":"2024-09-17T03:14:56.337801+0000","last_unstale":"2024-09-17T03:14:58.582562+0000","last_undegraded":"2024-09-17T03:14:58.582562+0000","last_fullsized":"2024-09-17T03:14:58.582562+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:11:00.835210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.340815+0000","last_change":"2024-09-17T03:14:55.328132+0000","last_active":"2024-09-17T03:14:57.340815+0000","last_peered":"2024-09-17T03:14:57.340815+0000","last_clean":"2024-09-17T03:14:57.340815+0000","last_became_active":"2024-09-17T03:14:55.327773+0000","last_became_peered":"2024-09-17T03:14:55.327773+0000","last_unstale":"2024-09-17T03:14:57.340815+0000","last_undegraded":"2024-09-17T03:14:57.340815+0000","last_fullsized":"2024-09-17T03:14:57.340815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:03:09.899391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.345152+0000","last_change":"2024-09-17T03:14:56.346331+0000","last_active":"2024-09-17T03:14:57.345152+0000","last_peered":"2024-09-17T03:14:57.345152+0000","last_clean":"2024-09-17T03:14:57.345152+0000","last_became_active":"2024-09-17T03:14:56.345971+0000","last_became_peered":"2024-09-17T03:14:56.345971+0000","last_unstale":"2024-09-17T03:14:57.345152+0000","last_undegraded":"2024-09-17T03:14:57.345152+0000","last_fullsized":"2024-09-17T03:14:57.345152+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:05.178403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583306+0000","last_change":"2024-09-17T03:14:55.334322+0000","last_active":"2024-09-17T03:14:58.583306+0000","last_peered":"2024-09-17T03:14:58.583306+0000","last_clean":"2024-09-17T03:14:58.583306+0000","last_became_active":"2024-09-17T03:14:55.333947+0000","last_became_peered":"2024-09-17T03:14:55.333947+0000","last_unstale":"2024-09-17T03:14:58.583306+0000","last_undegraded":"2024-09-17T03:14:58.583306+0000","last_fullsized":"2024-09-17T03:14:58.583306+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:44.089548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.196690+0000","last_change":"2024-09-17T03:14:56.345419+0000","last_active":"2024-09-17T03:14:58.196690+0000","last_peered":"2024-09-17T03:14:58.196690+0000","last_clean":"2024-09-17T03:14:58.196690+0000","last_became_active":"2024-09-17T03:14:56.343640+0000","last_became_peered":"2024-09-17T03:14:56.343640+0000","last_unstale":"2024-09-17T03:14:58.196690+0000","last_undegraded":"2024-09-17T03:14:58.196690+0000","last_fullsized":"2024-09-17T03:14:58.196690+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:34:21.864834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344278+0000","last_change":"2024-09-17T03:14:55.323288+0000","last_active":"2024-09-17T03:14:57.344278+0000","last_peered":"2024-09-17T03:14:57.344278+0000","last_clean":"2024-09-17T03:14:57.344278+0000","last_became_active":"2024-09-17T03:14:55.322723+0000","last_became_peered":"2024-09-17T03:14:55.322723+0000","last_unstale":"2024-09-17T03:14:57.344278+0000","last_undegraded":"2024-09-17T03:14:57.344278+0000","last_fullsized":"2024-09-17T03:14:57.344278+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:11:24.012880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.345860+0000","last_change":"2024-09-17T03:14:56.346227+0000","last_active":"2024-09-17T03:14:57.345860+0000","last_peered":"2024-09-17T03:14:57.345860+0000","last_clean":"2024-09-17T03:14:57.345860+0000","last_became_active":"2024-09-17T03:14:56.345908+0000","last_became_peered":"2024-09-17T03:14:56.345908+0000","last_unstale":"2024-09-17T03:14:57.345860+0000","last_undegraded":"2024-09-17T03:14:57.345860+0000","last_fullsized":"2024-09-17T03:14:57.345860+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:46:44.337263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344030+0000","last_change":"2024-09-17T03:14:55.325887+0000","last_active":"2024-09-17T03:14:57.344030+0000","last_peered":"2024-09-17T03:14:57.344030+0000","last_clean":"2024-09-17T03:14:57.344030+0000","last_became_active":"2024-09-17T03:14:55.325563+0000","last_became_peered":"2024-09-17T03:14:55.325563+0000","last_unstale":"2024-09-17T03:14:57.344030+0000","last_undegraded":"2024-09-17T03:14:57.344030+0000","last_fullsized":"2024-09-17T03:14:57.344030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:22.517631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.345969+0000","last_change":"2024-09-17T03:14:56.352353+0000","last_active":"2024-09-17T03:14:57.345969+0000","last_peered":"2024-09-17T03:14:57.345969+0000","last_clean":"2024-09-17T03:14:57.345969+0000","last_became_active":"2024-09-17T03:14:56.352011+0000","last_became_peered":"2024-09-17T03:14:56.352011+0000","last_unstale":"2024-09-17T03:14:57.345969+0000","last_undegraded":"2024-09-17T03:14:57.345969+0000","last_fullsized":"2024-09-17T03:14:57.345969+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:22:35.683440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.196557+0000","last_change":"2024-09-17T03:14:55.329954+0000","last_active":"2024-09-17T03:14:58.196557+0000","last_peered":"2024-09-17T03:14:58.196557+0000","last_clean":"2024-09-17T03:14:58.196557+0000","last_became_active":"2024-09-17T03:14:55.329631+0000","last_became_peered":"2024-09-17T03:14:55.329631+0000","last_unstale":"2024-09-17T03:14:58.196557+0000","last_undegraded":"2024-09-17T03:14:58.196557+0000","last_fullsized":"2024-09-17T03:14:58.196557+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:45.028465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.196825+0000","last_change":"2024-09-17T03:14:56.352353+0000","last_active":"2024-09-17T03:14:58.196825+0000","last_peered":"2024-09-17T03:14:58.196825+0000","last_clean":"2024-09-17T03:14:58.196825+0000","last_became_active":"2024-09-17T03:14:56.352041+0000","last_became_peered":"2024-09-17T03:14:56.352041+0000","last_unstale":"2024-09-17T03:14:58.196825+0000","last_undegraded":"2024-09-17T03:14:58.196825+0000","last_fullsized":"2024-09-17T03:14:58.196825+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:42:54.094479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":21,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583482+0000","last_change":"2024-09-17T03:14:55.332350+0000","last_active":"2024-09-17T03:14:58.583482+0000","last_peered":"2024-09-17T03:14:58.583482+0000","last_clean":"2024-09-17T03:14:58.583482+0000","last_became_active":"2024-09-17T03:14:55.332010+0000","last_became_peered":"2024-09-17T03:14:55.332010+0000","last_unstale":"2024-09-17T03:14:58.583482+0000","last_undegraded":"2024-09-17T03:14:58.583482+0000","last_fullsized":"2024-09-17T03:14:58.583482+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:19.084170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583141+0000","last_change":"2024-09-17T03:14:56.344117+0000","last_active":"2024-09-17T03:14:58.583141+0000","last_peered":"2024-09-17T03:14:58.583141+0000","last_clean":"2024-09-17T03:14:58.583141+0000","last_became_active":"2024-09-17T03:14:56.343821+0000","last_became_peered":"2024-09-17T03:14:56.343821+0000","last_unstale":"2024-09-17T03:14:58.583141+0000","last_undegraded":"2024-09-17T03:14:58.583141+0000","last_fullsized":"2024-09-17T03:14:58.583141+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:53:20.900685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583257+0000","last_change":"2024-09-17T03:14:55.333495+0000","last_active":"2024-09-17T03:14:58.583257+0000","last_peered":"2024-09-17T03:14:58.583257+0000","last_clean":"2024-09-17T03:14:58.583257+0000","last_became_active":"2024-09-17T03:14:55.332779+0000","last_became_peered":"2024-09-17T03:14:55.332779+0000","last_unstale":"2024-09-17T03:14:58.583257+0000","last_undegraded":"2024-09-17T03:14:58.583257+0000","last_fullsized":"2024-09-17T03:14:58.583257+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:37:19.084170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.345470+0000","last_change":"2024-09-17T03:14:56.342278+0000","last_active":"2024-09-17T03:14:57.345470+0000","last_peered":"2024-09-17T03:14:57.345470+0000","last_clean":"2024-09-17T03:14:57.345470+0000","last_became_active":"2024-09-17T03:14:56.341961+0000","last_became_peered":"2024-09-17T03:14:56.341961+0000","last_unstale":"2024-09-17T03:14:57.345470+0000","last_undegraded":"2024-09-17T03:14:57.345470+0000","last_fullsized":"2024-09-17T03:14:57.345470+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:41:19.135102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.345551+0000","last_change":"2024-09-17T03:14:55.326891+0000","last_active":"2024-09-17T03:14:57.345551+0000","last_peered":"2024-09-17T03:14:57.345551+0000","last_clean":"2024-09-17T03:14:57.345551+0000","last_became_active":"2024-09-17T03:14:55.326571+0000","last_became_peered":"2024-09-17T03:14:55.326571+0000","last_unstale":"2024-09-17T03:14:57.345551+0000","last_undegraded":"2024-09-17T03:14:57.345551+0000","last_fullsized":"2024-09-17T03:14:57.345551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:20:48.069665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583175+0000","last_change":"2024-09-17T03:14:56.341404+0000","last_active":"2024-09-17T03:14:58.583175+0000","last_peered":"2024-09-17T03:14:58.583175+0000","last_clean":"2024-09-17T03:14:58.583175+0000","last_became_active":"2024-09-17T03:14:56.341086+0000","last_became_peered":"2024-09-17T03:14:56.341086+0000","last_unstale":"2024-09-17T03:14:58.583175+0000","last_undegraded":"2024-09-17T03:14:58.583175+0000","last_fullsized":"2024-09-17T03:14:58.583175+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:29:42.736899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344859+0000","last_change":"2024-09-17T03:14:55.323707+0000","last_active":"2024-09-17T03:14:57.344859+0000","last_peered":"2024-09-17T03:14:57.344859+0000","last_clean":"2024-09-17T03:14:57.344859+0000","last_became_active":"2024-09-17T03:14:55.323141+0000","last_became_peered":"2024-09-17T03:14:55.323141+0000","last_unstale":"2024-09-17T03:14:57.344859+0000","last_undegraded":"2024-09-17T03:14:57.344859+0000","last_fullsized":"2024-09-17T03:14:57.344859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:36:42.070241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.196580+0000","last_change":"2024-09-17T03:14:56.347429+0000","last_active":"2024-09-17T03:14:58.196580+0000","last_peered":"2024-09-17T03:14:58.196580+0000","last_clean":"2024-09-17T03:14:58.196580+0000","last_became_active":"2024-09-17T03:14:56.347101+0000","last_became_peered":"2024-09-17T03:14:56.347101+0000","last_unstale":"2024-09-17T03:14:58.196580+0000","last_undegraded":"2024-09-17T03:14:58.196580+0000","last_fullsized":"2024-09-17T03:14:58.196580+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:03:03.864960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.340674+0000","last_change":"2024-09-17T03:14:55.330923+0000","last_active":"2024-09-17T03:14:57.340674+0000","last_peered":"2024-09-17T03:14:57.340674+0000","last_clean":"2024-09-17T03:14:57.340674+0000","last_became_active":"2024-09-17T03:14:55.330601+0000","last_became_peered":"2024-09-17T03:14:55.330601+0000","last_unstale":"2024-09-17T03:14:57.340674+0000","last_undegraded":"2024-09-17T03:14:57.340674+0000","last_fullsized":"2024-09-17T03:14:57.340674+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:21.326711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.196773+0000","last_change":"2024-09-17T03:14:56.343137+0000","last_active":"2024-09-17T03:14:58.196773+0000","last_peered":"2024-09-17T03:14:58.196773+0000","last_clean":"2024-09-17T03:14:58.196773+0000","last_became_active":"2024-09-17T03:14:56.342838+0000","last_became_peered":"2024-09-17T03:14:56.342838+0000","last_unstale":"2024-09-17T03:14:58.196773+0000","last_undegraded":"2024-09-17T03:14:58.196773+0000","last_fullsized":"2024-09-17T03:14:58.196773+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:14.387686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341150+0000","last_change":"2024-09-17T03:14:55.331984+0000","last_active":"2024-09-17T03:14:57.341150+0000","last_peered":"2024-09-17T03:14:57.341150+0000","last_clean":"2024-09-17T03:14:57.341150+0000","last_became_active":"2024-09-17T03:14:55.331543+0000","last_became_peered":"2024-09-17T03:14:55.331543+0000","last_unstale":"2024-09-17T03:14:57.341150+0000","last_undegraded":"2024-09-17T03:14:57.341150+0000","last_fullsized":"2024-09-17T03:14:57.341150+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:06:37.330728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583969+0000","last_change":"2024-09-17T03:14:56.336542+0000","last_active":"2024-09-17T03:14:58.583969+0000","last_peered":"2024-09-17T03:14:58.583969+0000","last_clean":"2024-09-17T03:14:58.583969+0000","last_became_active":"2024-09-17T03:14:56.336216+0000","last_became_peered":"2024-09-17T03:14:56.336216+0000","last_unstale":"2024-09-17T03:14:58.583969+0000","last_undegraded":"2024-09-17T03:14:58.583969+0000","last_fullsized":"2024-09-17T03:14:58.583969+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:56:14.690251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.582953+0000","last_change":"2024-09-17T03:14:55.335726+0000","last_active":"2024-09-17T03:14:58.582953+0000","last_peered":"2024-09-17T03:14:58.582953+0000","last_clean":"2024-09-17T03:14:58.582953+0000","last_became_active":"2024-09-17T03:14:55.335268+0000","last_became_peered":"2024-09-17T03:14:55.335268+0000","last_unstale":"2024-09-17T03:14:58.582953+0000","last_undegraded":"2024-09-17T03:14:58.582953+0000","last_fullsized":"2024-09-17T03:14:58.582953+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:13.660225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344275+0000","last_change":"2024-09-17T03:14:56.336628+0000","last_active":"2024-09-17T03:14:57.344275+0000","last_peered":"2024-09-17T03:14:57.344275+0000","last_clean":"2024-09-17T03:14:57.344275+0000","last_became_active":"2024-09-17T03:14:56.336330+0000","last_became_peered":"2024-09-17T03:14:56.336330+0000","last_unstale":"2024-09-17T03:14:57.344275+0000","last_undegraded":"2024-09-17T03:14:57.344275+0000","last_fullsized":"2024-09-17T03:14:57.344275+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:02:13.758412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.196961+0000","last_change":"2024-09-17T03:14:55.324166+0000","last_active":"2024-09-17T03:14:58.196961+0000","last_peered":"2024-09-17T03:14:58.196961+0000","last_clean":"2024-09-17T03:14:58.196961+0000","last_became_active":"2024-09-17T03:14:55.323844+0000","last_became_peered":"2024-09-17T03:14:55.323844+0000","last_unstale":"2024-09-17T03:14:58.196961+0000","last_undegraded":"2024-09-17T03:14:58.196961+0000","last_fullsized":"2024-09-17T03:14:58.196961+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:27:53.850399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341186+0000","last_change":"2024-09-17T03:14:56.337913+0000","last_active":"2024-09-17T03:14:57.341186+0000","last_peered":"2024-09-17T03:14:57.341186+0000","last_clean":"2024-09-17T03:14:57.341186+0000","last_became_active":"2024-09-17T03:14:56.337610+0000","last_became_peered":"2024-09-17T03:14:56.337610+0000","last_unstale":"2024-09-17T03:14:57.341186+0000","last_undegraded":"2024-09-17T03:14:57.341186+0000","last_fullsized":"2024-09-17T03:14:57.341186+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:47:08.606153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.584043+0000","last_change":"2024-09-17T03:14:55.333099+0000","last_active":"2024-09-17T03:14:58.584043+0000","last_peered":"2024-09-17T03:14:58.584043+0000","last_clean":"2024-09-17T03:14:58.584043+0000","last_became_active":"2024-09-17T03:14:55.332738+0000","last_became_peered":"2024-09-17T03:14:55.332738+0000","last_unstale":"2024-09-17T03:14:58.584043+0000","last_undegraded":"2024-09-17T03:14:58.584043+0000","last_fullsized":"2024-09-17T03:14:58.584043+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:34:50.915622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583066+0000","last_change":"2024-09-17T03:14:56.340119+0000","last_active":"2024-09-17T03:14:58.583066+0000","last_peered":"2024-09-17T03:14:58.583066+0000","last_clean":"2024-09-17T03:14:58.583066+0000","last_became_active":"2024-09-17T03:14:56.339795+0000","last_became_peered":"2024-09-17T03:14:56.339795+0000","last_unstale":"2024-09-17T03:14:58.583066+0000","last_undegraded":"2024-09-17T03:14:58.583066+0000","last_fullsized":"2024-09-17T03:14:58.583066+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:38:12.850097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344218+0000","last_change":"2024-09-17T03:14:55.327065+0000","last_active":"2024-09-17T03:14:57.344218+0000","last_peered":"2024-09-17T03:14:57.344218+0000","last_clean":"2024-09-17T03:14:57.344218+0000","last_became_active":"2024-09-17T03:14:55.326774+0000","last_became_peered":"2024-09-17T03:14:55.326774+0000","last_unstale":"2024-09-17T03:14:57.344218+0000","last_undegraded":"2024-09-17T03:14:57.344218+0000","last_fullsized":"2024-09-17T03:14:57.344218+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:37.138244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344009+0000","last_change":"2024-09-17T03:14:56.335950+0000","last_active":"2024-09-17T03:14:57.344009+0000","last_peered":"2024-09-17T03:14:57.344009+0000","last_clean":"2024-09-17T03:14:57.344009+0000","last_became_active":"2024-09-17T03:14:56.335641+0000","last_became_peered":"2024-09-17T03:14:56.335641+0000","last_unstale":"2024-09-17T03:14:57.344009+0000","last_undegraded":"2024-09-17T03:14:57.344009+0000","last_fullsized":"2024-09-17T03:14:57.344009+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:43:03.609971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583828+0000","last_change":"2024-09-17T03:14:55.335625+0000","last_active":"2024-09-17T03:14:58.583828+0000","last_peered":"2024-09-17T03:14:58.583828+0000","last_clean":"2024-09-17T03:14:58.583828+0000","last_became_active":"2024-09-17T03:14:55.335317+0000","last_became_peered":"2024-09-17T03:14:55.335317+0000","last_unstale":"2024-09-17T03:14:58.583828+0000","last_undegraded":"2024-09-17T03:14:58.583828+0000","last_fullsized":"2024-09-17T03:14:58.583828+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:57:47.254999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.582984+0000","last_change":"2024-09-17T03:14:56.346898+0000","last_active":"2024-09-17T03:14:58.582984+0000","last_peered":"2024-09-17T03:14:58.582984+0000","last_clean":"2024-09-17T03:14:58.582984+0000","last_became_active":"2024-09-17T03:14:56.346603+0000","last_became_peered":"2024-09-17T03:14:56.346603+0000","last_unstale":"2024-09-17T03:14:58.582984+0000","last_undegraded":"2024-09-17T03:14:58.582984+0000","last_fullsized":"2024-09-17T03:14:58.582984+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:34:03.584090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583828+0000","last_change":"2024-09-17T03:14:55.328041+0000","last_active":"2024-09-17T03:14:58.583828+0000","last_peered":"2024-09-17T03:14:58.583828+0000","last_clean":"2024-09-17T03:14:58.583828+0000","last_became_active":"2024-09-17T03:14:55.327533+0000","last_became_peered":"2024-09-17T03:14:55.327533+0000","last_unstale":"2024-09-17T03:14:58.583828+0000","last_undegraded":"2024-09-17T03:14:58.583828+0000","last_fullsized":"2024-09-17T03:14:58.583828+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:49:21.326711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583649+0000","last_change":"2024-09-17T03:14:56.336599+0000","last_active":"2024-09-17T03:14:58.583649+0000","last_peered":"2024-09-17T03:14:58.583649+0000","last_clean":"2024-09-17T03:14:58.583649+0000","last_became_active":"2024-09-17T03:14:56.336243+0000","last_became_peered":"2024-09-17T03:14:56.336243+0000","last_unstale":"2024-09-17T03:14:58.583649+0000","last_undegraded":"2024-09-17T03:14:58.583649+0000","last_fullsized":"2024-09-17T03:14:58.583649+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:45.095341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.196367+0000","last_change":"2024-09-17T03:14:55.324984+0000","last_active":"2024-09-17T03:14:58.196367+0000","last_peered":"2024-09-17T03:14:58.196367+0000","last_clean":"2024-09-17T03:14:58.196367+0000","last_became_active":"2024-09-17T03:14:55.324696+0000","last_became_peered":"2024-09-17T03:14:55.324696+0000","last_unstale":"2024-09-17T03:14:58.196367+0000","last_undegraded":"2024-09-17T03:14:58.196367+0000","last_fullsized":"2024-09-17T03:14:58.196367+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:56:10.371792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341211+0000","last_change":"2024-09-17T03:14:56.338902+0000","last_active":"2024-09-17T03:14:57.341211+0000","last_peered":"2024-09-17T03:14:57.341211+0000","last_clean":"2024-09-17T03:14:57.341211+0000","last_became_active":"2024-09-17T03:14:56.338504+0000","last_became_peered":"2024-09-17T03:14:56.338504+0000","last_unstale":"2024-09-17T03:14:57.341211+0000","last_undegraded":"2024-09-17T03:14:57.341211+0000","last_fullsized":"2024-09-17T03:14:57.341211+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:16.330515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583375+0000","last_change":"2024-09-17T03:14:55.328618+0000","last_active":"2024-09-17T03:14:58.583375+0000","last_peered":"2024-09-17T03:14:58.583375+0000","last_clean":"2024-09-17T03:14:58.583375+0000","last_became_active":"2024-09-17T03:14:55.328321+0000","last_became_peered":"2024-09-17T03:14:55.328321+0000","last_unstale":"2024-09-17T03:14:58.583375+0000","last_undegraded":"2024-09-17T03:14:58.583375+0000","last_fullsized":"2024-09-17T03:14:58.583375+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:30:39.180289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.196902+0000","last_change":"2024-09-17T03:14:56.341818+0000","last_active":"2024-09-17T03:14:58.196902+0000","last_peered":"2024-09-17T03:14:58.196902+0000","last_clean":"2024-09-17T03:14:58.196902+0000","last_became_active":"2024-09-17T03:14:56.341526+0000","last_became_peered":"2024-09-17T03:14:56.341526+0000","last_unstale":"2024-09-17T03:14:58.196902+0000","last_undegraded":"2024-09-17T03:14:58.196902+0000","last_fullsized":"2024-09-17T03:14:58.196902+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:12.181812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.524745+0000","last_change":"2024-09-17T03:14:55.325076+0000","last_active":"2024-09-17T03:14:57.524745+0000","last_peered":"2024-09-17T03:14:57.524745+0000","last_clean":"2024-09-17T03:14:57.524745+0000","last_became_active":"2024-09-17T03:14:55.324738+0000","last_became_peered":"2024-09-17T03:14:55.324738+0000","last_unstale":"2024-09-17T03:14:57.524745+0000","last_undegraded":"2024-09-17T03:14:57.524745+0000","last_fullsized":"2024-09-17T03:14:57.524745+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:26:33.226093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.341372+0000","last_change":"2024-09-17T03:14:56.348451+0000","last_active":"2024-09-17T03:14:57.341372+0000","last_peered":"2024-09-17T03:14:57.341372+0000","last_clean":"2024-09-17T03:14:57.341372+0000","last_became_active":"2024-09-17T03:14:56.348169+0000","last_became_peered":"2024-09-17T03:14:56.348169+0000","last_unstale":"2024-09-17T03:14:57.341372+0000","last_undegraded":"2024-09-17T03:14:57.341372+0000","last_fullsized":"2024-09-17T03:14:57.341372+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:11:47.114511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583621+0000","last_change":"2024-09-17T03:14:55.327511+0000","last_active":"2024-09-17T03:14:58.583621+0000","last_peered":"2024-09-17T03:14:58.583621+0000","last_clean":"2024-09-17T03:14:58.583621+0000","last_became_active":"2024-09-17T03:14:55.327195+0000","last_became_peered":"2024-09-17T03:14:55.327195+0000","last_unstale":"2024-09-17T03:14:58.583621+0000","last_undegraded":"2024-09-17T03:14:58.583621+0000","last_fullsized":"2024-09-17T03:14:58.583621+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:14:13.660225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583561+0000","last_change":"2024-09-17T03:14:56.337722+0000","last_active":"2024-09-17T03:14:58.583561+0000","last_peered":"2024-09-17T03:14:58.583561+0000","last_clean":"2024-09-17T03:14:58.583561+0000","last_became_active":"2024-09-17T03:14:56.337353+0000","last_became_peered":"2024-09-17T03:14:56.337353+0000","last_unstale":"2024-09-17T03:14:58.583561+0000","last_undegraded":"2024-09-17T03:14:58.583561+0000","last_fullsized":"2024-09-17T03:14:58.583561+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:24:46.034257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583453+0000","last_change":"2024-09-17T03:14:55.328438+0000","last_active":"2024-09-17T03:14:58.583453+0000","last_peered":"2024-09-17T03:14:58.583453+0000","last_clean":"2024-09-17T03:14:58.583453+0000","last_became_active":"2024-09-17T03:14:55.328158+0000","last_became_peered":"2024-09-17T03:14:55.328158+0000","last_unstale":"2024-09-17T03:14:58.583453+0000","last_undegraded":"2024-09-17T03:14:58.583453+0000","last_fullsized":"2024-09-17T03:14:58.583453+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:19:28.098924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.345862+0000","last_change":"2024-09-17T03:14:56.350030+0000","last_active":"2024-09-17T03:14:57.345862+0000","last_peered":"2024-09-17T03:14:57.345862+0000","last_clean":"2024-09-17T03:14:57.345862+0000","last_became_active":"2024-09-17T03:14:56.349711+0000","last_became_peered":"2024-09-17T03:14:56.349711+0000","last_unstale":"2024-09-17T03:14:57.345862+0000","last_undegraded":"2024-09-17T03:14:57.345862+0000","last_fullsized":"2024-09-17T03:14:57.345862+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:24:34.658175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583313+0000","last_change":"2024-09-17T03:14:55.330495+0000","last_active":"2024-09-17T03:14:58.583313+0000","last_peered":"2024-09-17T03:14:58.583313+0000","last_clean":"2024-09-17T03:14:58.583313+0000","last_became_active":"2024-09-17T03:14:55.330159+0000","last_became_peered":"2024-09-17T03:14:55.330159+0000","last_unstale":"2024-09-17T03:14:58.583313+0000","last_undegraded":"2024-09-17T03:14:58.583313+0000","last_fullsized":"2024-09-17T03:14:58.583313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:55.212014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.345084+0000","last_change":"2024-09-17T03:14:56.351627+0000","last_active":"2024-09-17T03:14:57.345084+0000","last_peered":"2024-09-17T03:14:57.345084+0000","last_clean":"2024-09-17T03:14:57.345084+0000","last_became_active":"2024-09-17T03:14:56.351236+0000","last_became_peered":"2024-09-17T03:14:56.351236+0000","last_unstale":"2024-09-17T03:14:57.345084+0000","last_undegraded":"2024-09-17T03:14:57.345084+0000","last_fullsized":"2024-09-17T03:14:57.345084+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:40:33.059245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583444+0000","last_change":"2024-09-17T03:14:55.331670+0000","last_active":"2024-09-17T03:14:58.583444+0000","last_peered":"2024-09-17T03:14:58.583444+0000","last_clean":"2024-09-17T03:14:58.583444+0000","last_became_active":"2024-09-17T03:14:55.331359+0000","last_became_peered":"2024-09-17T03:14:55.331359+0000","last_unstale":"2024-09-17T03:14:58.583444+0000","last_undegraded":"2024-09-17T03:14:58.583444+0000","last_fullsized":"2024-09-17T03:14:58.583444+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:17:55.212014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.196567+0000","last_change":"2024-09-17T03:14:56.341402+0000","last_active":"2024-09-17T03:14:58.196567+0000","last_peered":"2024-09-17T03:14:58.196567+0000","last_clean":"2024-09-17T03:14:58.196567+0000","last_became_active":"2024-09-17T03:14:56.341095+0000","last_became_peered":"2024-09-17T03:14:56.341095+0000","last_unstale":"2024-09-17T03:14:58.196567+0000","last_undegraded":"2024-09-17T03:14:58.196567+0000","last_fullsized":"2024-09-17T03:14:58.196567+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:41:29.183616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:57.344951+0000","last_change":"2024-09-17T03:14:55.330612+0000","last_active":"2024-09-17T03:14:57.344951+0000","last_peered":"2024-09-17T03:14:57.344951+0000","last_clean":"2024-09-17T03:14:57.344951+0000","last_became_active":"2024-09-17T03:14:55.330302+0000","last_became_peered":"2024-09-17T03:14:55.330302+0000","last_unstale":"2024-09-17T03:14:57.344951+0000","last_undegraded":"2024-09-17T03:14:57.344951+0000","last_fullsized":"2024-09-17T03:14:57.344951+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:04:59.075419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.583882+0000","last_change":"2024-09-17T03:14:56.337604+0000","last_active":"2024-09-17T03:14:58.583882+0000","last_peered":"2024-09-17T03:14:58.583882+0000","last_clean":"2024-09-17T03:14:58.583882+0000","last_became_active":"2024-09-17T03:14:56.337283+0000","last_became_peered":"2024-09-17T03:14:56.337283+0000","last_unstale":"2024-09-17T03:14:58.583882+0000","last_undegraded":"2024-09-17T03:14:58.583882+0000","last_fullsized":"2024-09-17T03:14:58.583882+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:55.307935+0000","last_clean_scrub_stamp":"2024-09-17T03:14:55.307935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:27:54.856191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.196985+0000","last_change":"2024-09-17T03:14:55.322959+0000","last_active":"2024-09-17T03:14:58.196985+0000","last_peered":"2024-09-17T03:14:58.196985+0000","last_clean":"2024-09-17T03:14:58.196985+0000","last_became_active":"2024-09-17T03:14:55.322475+0000","last_became_peered":"2024-09-17T03:14:55.322475+0000","last_unstale":"2024-09-17T03:14:58.196985+0000","last_undegraded":"2024-09-17T03:14:58.196985+0000","last_fullsized":"2024-09-17T03:14:58.196985+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:54.302142+0000","last_clean_scrub_stamp":"2024-09-17T03:14:54.302142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:31:57.870235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":129,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-17T03:14:58.582801+0000","last_change":"2024-09-17T03:14:04.862167+0000","last_active":"2024-09-17T03:14:58.582801+0000","last_peered":"2024-09-17T03:14:58.582801+0000","last_clean":"2024-09-17T03:14:58.582801+0000","last_became_active":"2024-09-17T03:14:04.861731+0000","last_became_peered":"2024-09-17T03:14:04.861731+0000","last_unstale":"2024-09-17T03:14:58.582801+0000","last_undegraded":"2024-09-17T03:14:58.582801+0000","last_fullsized":"2024-09-17T03:14:58.582801+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_clean_scrub_stamp":"2024-09-17T03:14:03.847721+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:32.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509457,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":380928,"data_stored":145329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":146919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":974848,"data_stored":736257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":146083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509457,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":145341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509457,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":974848,"data_stored":735745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509456,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":145397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":146941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:15:04.161 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T03:15:04.162 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T03:15:04.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.283+0000 7f83a435f640 1 -- 172.21.15.87:0/2552027026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8388004000 msgr2=0x7f8388004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:04.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.283+0000 7f83a435f640 1 --2- 172.21.15.87:0/2552027026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8388004000 0x7f8388004800 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f839c071be0 tx=0x7f8388005830 comp rx=0 tx=0).stop 2024-09-17T03:15:04.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.283+0000 7f83a28fc640 1 -- 172.21.15.87:0/2552027026 reap_dead start 2024-09-17T03:15:04.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.284+0000 7f83a435f640 1 -- 172.21.15.87:0/2552027026 shutdown_connections 2024-09-17T03:15:04.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.284+0000 7f83a435f640 1 -- 172.21.15.87:0/2552027026 >> 172.21.15.87:0/2552027026 conn(0x7f839c069120 msgr2=0x7f839c069530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:04.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.284+0000 7f83a435f640 1 -- 172.21.15.87:0/2552027026 shutdown_connections 2024-09-17T03:15:04.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.284+0000 7f83a435f640 1 -- 172.21.15.87:0/2552027026 wait complete. 2024-09-17T03:15:04.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.285+0000 7f83a435f640 1 Processor -- start 2024-09-17T03:15:04.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.285+0000 7f83a435f640 1 -- start start 2024-09-17T03:15:04.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.287+0000 7f83a435f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8388004000 0x7f839c0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:04.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.287+0000 7f83a435f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f839c0b4460 0x7f839c15d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:04.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.287+0000 7f83a435f640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f839c15dba0 0x7f839c158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:04.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.287+0000 7f83a435f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f839c0d0880 con 0x7f839c0b4460 2024-09-17T03:15:04.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.287+0000 7f83a435f640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f839c0d0700 con 0x7f8388004000 2024-09-17T03:15:04.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.287+0000 7f83a435f640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f839c0d0a00 con 0x7f839c15dba0 2024-09-17T03:15:04.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.287+0000 7f83a18fa640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f839c0b4460 0x7f839c15d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:04.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.287+0000 7f83a18fa640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f839c0b4460 0x7f839c15d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40084/0 (socket says 172.21.15.87:40084) 2024-09-17T03:15:04.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.287+0000 7f83a18fa640 1 -- 172.21.15.87:0/4218401930 learned_addr learned my addr 172.21.15.87:0/4218401930 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:04.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.287+0000 7f83a20fb640 1 --2- 172.21.15.87:0/4218401930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8388004000 0x7f839c0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:04.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.287+0000 7f83a28fc640 1 --2- 172.21.15.87:0/4218401930 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f839c15dba0 0x7f839c158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:04.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.287+0000 7f83a18fa640 1 -- 172.21.15.87:0/4218401930 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f839c15dba0 msgr2=0x7f839c158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:04.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.287+0000 7f83a18fa640 1 --2- 172.21.15.87:0/4218401930 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f839c15dba0 0x7f839c158f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:04.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.287+0000 7f83a18fa640 1 -- 172.21.15.87:0/4218401930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8388004000 msgr2=0x7f839c0b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:04.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.287+0000 7f83a18fa640 1 --2- 172.21.15.87:0/4218401930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8388004000 0x7f839c0b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:04.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.287+0000 7f83a18fa640 1 -- 172.21.15.87:0/4218401930 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c15fe10 con 0x7f839c0b4460 2024-09-17T03:15:04.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.287+0000 7f83a20fb640 1 --2- 172.21.15.87:0/4218401930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8388004000 0x7f839c0b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:15:04.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.287+0000 7f83a28fc640 1 --2- 172.21.15.87:0/4218401930 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f839c15dba0 0x7f839c158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:15:04.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.287+0000 7f83a18fa640 1 --2- 172.21.15.87:0/4218401930 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f839c0b4460 0x7f839c15d660 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f838c00c560 tx=0x7f838c0013c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:04.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.287+0000 7f8392ffd640 1 -- 172.21.15.87:0/4218401930 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c042070 con 0x7f839c0b4460 2024-09-17T03:15:04.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.287+0000 7f8392ffd640 1 -- 172.21.15.87:0/4218401930 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f838c0035f0 con 0x7f839c0b4460 2024-09-17T03:15:04.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.287+0000 7f83a435f640 1 -- 172.21.15.87:0/4218401930 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f839c15d0b0 con 0x7f839c0b4460 2024-09-17T03:15:04.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.287+0000 7f8392ffd640 1 -- 172.21.15.87:0/4218401930 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f838c039070 con 0x7f839c0b4460 2024-09-17T03:15:04.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.287+0000 7f83a435f640 1 -- 172.21.15.87:0/4218401930 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f839c160390 con 0x7f839c0b4460 2024-09-17T03:15:04.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.289+0000 7f83a435f640 1 -- 172.21.15.87:0/4218401930 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f839c15d3f0 con 0x7f839c0b4460 2024-09-17T03:15:04.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.289+0000 7f8392ffd640 1 -- 172.21.15.87:0/4218401930 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f838c045070 con 0x7f839c0b4460 2024-09-17T03:15:04.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.290+0000 7f8392ffd640 1 --2- 172.21.15.87:0/4218401930 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8380078bc0 0x7f838007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:04.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.290+0000 7f8392ffd640 1 -- 172.21.15.87:0/4218401930 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f838c0ba7b0 con 0x7f839c0b4460 2024-09-17T03:15:04.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.290+0000 7f83a20fb640 1 --2- 172.21.15.87:0/4218401930 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8380078bc0 0x7f838007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:04.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.291+0000 7f83a20fb640 1 --2- 172.21.15.87:0/4218401930 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8380078bc0 0x7f838007b0d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8398006990 tx=0x7f8398006ac0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:04.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.294+0000 7f8392ffd640 1 -- 172.21.15.87:0/4218401930 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f839c0cd440 con 0x7f839c0b4460 2024-09-17T03:15:04.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.547+0000 7f83a435f640 1 -- 172.21.15.87:0/4218401930 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f839c0d0f90 con 0x7f839c0b4460 2024-09-17T03:15:04.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.552+0000 7f8392ffd640 1 -- 172.21.15.87:0/4218401930 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f838c082cd0 con 0x7f839c0b4460 2024-09-17T03:15:04.550 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-09-17T03:15:04.551 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDH8+hmLskNLhAAd2M5ynGP8W6Pim11F18wYA== 2024-09-17T03:15:04.551 INFO:teuthology.orchestra.run.smithi087.stdout: caps mds = "allow" 2024-09-17T03:15:04.551 INFO:teuthology.orchestra.run.smithi087.stdout: caps mon = "allow r" 2024-09-17T03:15:04.551 INFO:teuthology.orchestra.run.smithi087.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T03:15:04.551 INFO:teuthology.orchestra.run.smithi087.stderr:updated caps for client.0 2024-09-17T03:15:04.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.554+0000 7f83a435f640 1 -- 172.21.15.87:0/4218401930 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8380078bc0 msgr2=0x7f838007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:04.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.554+0000 7f83a435f640 1 --2- 172.21.15.87:0/4218401930 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8380078bc0 0x7f838007b0d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8398006990 tx=0x7f8398006ac0 comp rx=0 tx=0).stop 2024-09-17T03:15:04.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.554+0000 7f83a435f640 1 -- 172.21.15.87:0/4218401930 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f839c0b4460 msgr2=0x7f839c15d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:04.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.554+0000 7f83a435f640 1 --2- 172.21.15.87:0/4218401930 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f839c0b4460 0x7f839c15d660 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f838c00c560 tx=0x7f838c0013c0 comp rx=0 tx=0).stop 2024-09-17T03:15:04.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.554+0000 7f83a435f640 1 -- 172.21.15.87:0/4218401930 shutdown_connections 2024-09-17T03:15:04.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.554+0000 7f83a435f640 1 --2- 172.21.15.87:0/4218401930 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8380078bc0 0x7f838007b0d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:04.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.554+0000 7f83a435f640 1 --2- 172.21.15.87:0/4218401930 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f839c15dba0 0x7f839c158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:04.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.554+0000 7f83a435f640 1 --2- 172.21.15.87:0/4218401930 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f839c0b4460 0x7f839c15d660 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:04.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.554+0000 7f83a435f640 1 --2- 172.21.15.87:0/4218401930 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8388004000 0x7f839c0b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:04.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.554+0000 7f83a435f640 1 -- 172.21.15.87:0/4218401930 >> 172.21.15.87:0/4218401930 conn(0x7f839c069120 msgr2=0x7f839c0b79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:04.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.554+0000 7f83a435f640 1 -- 172.21.15.87:0/4218401930 shutdown_connections 2024-09-17T03:15:04.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.554+0000 7f83a435f640 1 -- 172.21.15.87:0/4218401930 wait complete. 2024-09-17T03:15:04.564 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:15:04.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.678+0000 7fde6eb9d640 1 -- 172.21.15.87:0/3327358799 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde68074350 msgr2=0x7fde68074890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:04.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.678+0000 7fde6eb9d640 1 --2- 172.21.15.87:0/3327358799 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde68074350 0x7fde68074890 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fde5c009830 tx=0x7fde5c02a510 comp rx=0 tx=0).stop 2024-09-17T03:15:04.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.678+0000 7fde6eb9d640 1 -- 172.21.15.87:0/3327358799 shutdown_connections 2024-09-17T03:15:04.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.678+0000 7fde6eb9d640 1 --2- 172.21.15.87:0/3327358799 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde68074350 0x7fde68074890 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:04.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.678+0000 7fde6eb9d640 1 -- 172.21.15.87:0/3327358799 >> 172.21.15.87:0/3327358799 conn(0x7fde680d10c0 msgr2=0x7fde680d14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:04.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.678+0000 7fde6eb9d640 1 -- 172.21.15.87:0/3327358799 shutdown_connections 2024-09-17T03:15:04.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.678+0000 7fde6eb9d640 1 -- 172.21.15.87:0/3327358799 wait complete. 2024-09-17T03:15:04.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.680+0000 7fde6eb9d640 1 Processor -- start 2024-09-17T03:15:04.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.680+0000 7fde6eb9d640 1 -- start start 2024-09-17T03:15:04.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.681+0000 7fde6eb9d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde68071e30 0x7fde680c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:04.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.681+0000 7fde6eb9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde6815b440 0x7fde680c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:04.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.681+0000 7fde6eb9d640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fde680a10a0 0x7fde680a14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:04.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.681+0000 7fde6eb9d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fde68071220 con 0x7fde6815b440 2024-09-17T03:15:04.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.681+0000 7fde6eb9d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fde680710a0 con 0x7fde68071e30 2024-09-17T03:15:04.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.681+0000 7fde6eb9d640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fde680713a0 con 0x7fde680a10a0 2024-09-17T03:15:04.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.681+0000 7fde6d39a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde6815b440 0x7fde680c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:04.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.681+0000 7fde6d39a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde6815b440 0x7fde680c4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:40104/0 (socket says 172.21.15.87:40104) 2024-09-17T03:15:04.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.681+0000 7fde6d39a640 1 -- 172.21.15.87:0/3487516750 learned_addr learned my addr 172.21.15.87:0/3487516750 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:04.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.682+0000 7fde6db9b640 1 --2- 172.21.15.87:0/3487516750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde68071e30 0x7fde680c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:04.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.682+0000 7fde6d39a640 1 -- 172.21.15.87:0/3487516750 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fde680a10a0 msgr2=0x7fde680a14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:04.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.682+0000 7fde6d39a640 1 --2- 172.21.15.87:0/3487516750 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fde680a10a0 0x7fde680a14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:04.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.682+0000 7fde6d39a640 1 -- 172.21.15.87:0/3487516750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde68071e30 msgr2=0x7fde680c71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:04.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.682+0000 7fde6d39a640 1 --2- 172.21.15.87:0/3487516750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde68071e30 0x7fde680c71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:04.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.682+0000 7fde6d39a640 1 -- 172.21.15.87:0/3487516750 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde680c7730 con 0x7fde6815b440 2024-09-17T03:15:04.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.682+0000 7fde6db9b640 1 --2- 172.21.15.87:0/3487516750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde68071e30 0x7fde680c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:15:04.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.682+0000 7fde6d39a640 1 --2- 172.21.15.87:0/3487516750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde6815b440 0x7fde680c4c80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fde5c02ab00 tx=0x7fde5c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:04.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.682+0000 7fde667fc640 1 -- 172.21.15.87:0/3487516750 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5c031d60 con 0x7fde6815b440 2024-09-17T03:15:04.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.682+0000 7fde667fc640 1 -- 172.21.15.87:0/3487516750 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde5c033e10 con 0x7fde6815b440 2024-09-17T03:15:04.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.682+0000 7fde6eb9d640 1 -- 172.21.15.87:0/3487516750 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde680b5560 con 0x7fde6815b440 2024-09-17T03:15:04.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.682+0000 7fde667fc640 1 -- 172.21.15.87:0/3487516750 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5c005260 con 0x7fde6815b440 2024-09-17T03:15:04.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.682+0000 7fde6eb9d640 1 -- 172.21.15.87:0/3487516750 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde680b5b20 con 0x7fde6815b440 2024-09-17T03:15:04.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.683+0000 7fde6eb9d640 1 -- 172.21.15.87:0/3487516750 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde680718d0 con 0x7fde6815b440 2024-09-17T03:15:04.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.684+0000 7fde667fc640 1 -- 172.21.15.87:0/3487516750 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fde5c005400 con 0x7fde6815b440 2024-09-17T03:15:04.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.684+0000 7fde667fc640 1 --2- 172.21.15.87:0/3487516750 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fde54078b90 0x7fde5407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:04.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.684+0000 7fde667fc640 1 -- 172.21.15.87:0/3487516750 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fde5c0bc150 con 0x7fde6815b440 2024-09-17T03:15:04.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.684+0000 7fde6db9b640 1 --2- 172.21.15.87:0/3487516750 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fde54078b90 0x7fde5407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:04.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.685+0000 7fde6db9b640 1 --2- 172.21.15.87:0/3487516750 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fde54078b90 0x7fde5407b0a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fde58007e90 tx=0x7fde580073a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:04.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.689+0000 7fde667fc640 1 -- 172.21.15.87:0/3487516750 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fde5c03f030 con 0x7fde6815b440 2024-09-17T03:15:04.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.895+0000 7fde6eb9d640 1 -- 172.21.15.87:0/3487516750 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde680b7040 con 0x7fde6815b440 2024-09-17T03:15:04.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.895+0000 7fde667fc640 1 -- 172.21.15.87:0/3487516750 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+5226 (secure 0 0 0) 0x7fde5c0844f0 con 0x7fde6815b440 2024-09-17T03:15:04.894 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:04.894 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":13,"btime":"2024-09-17T03:14:58:337521+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6836/1081794135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1081794135},{"type":"v1","addr":"172.21.15.149:6836","nonce":1081794135}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2152665509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6835","nonce":2152665509},{"type":"v1","addr":"172.21.15.149:6837","nonce":2152665509}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/704821751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":704821751},{"type":"v1","addr":"172.21.15.87:6835","nonce":704821751}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:14:55.675616+0000","modified":"2024-09-17T03:14:57.683823+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5004},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5004":{"gid":5004,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6834/1243349323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":1243349323},{"type":"v1","addr":"172.21.15.87:6834","nonce":1243349323}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5004,"qdb_cluster":[5004]},"id":2}]} 2024-09-17T03:15:04.895 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 13 2024-09-17T03:15:04.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.897+0000 7fde6eb9d640 1 -- 172.21.15.87:0/3487516750 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fde54078b90 msgr2=0x7fde5407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:04.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.897+0000 7fde6eb9d640 1 --2- 172.21.15.87:0/3487516750 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fde54078b90 0x7fde5407b0a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fde58007e90 tx=0x7fde580073a0 comp rx=0 tx=0).stop 2024-09-17T03:15:04.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.897+0000 7fde6eb9d640 1 -- 172.21.15.87:0/3487516750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde6815b440 msgr2=0x7fde680c4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:04.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.897+0000 7fde6eb9d640 1 --2- 172.21.15.87:0/3487516750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde6815b440 0x7fde680c4c80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fde5c02ab00 tx=0x7fde5c033c20 comp rx=0 tx=0).stop 2024-09-17T03:15:04.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.897+0000 7fde6eb9d640 1 -- 172.21.15.87:0/3487516750 shutdown_connections 2024-09-17T03:15:04.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.897+0000 7fde6eb9d640 1 --2- 172.21.15.87:0/3487516750 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fde54078b90 0x7fde5407b0a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:04.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.897+0000 7fde6eb9d640 1 --2- 172.21.15.87:0/3487516750 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fde680a10a0 0x7fde680a14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:04.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.897+0000 7fde6eb9d640 1 --2- 172.21.15.87:0/3487516750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde6815b440 0x7fde680c4c80 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:04.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.897+0000 7fde6eb9d640 1 --2- 172.21.15.87:0/3487516750 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fde68071e30 0x7fde680c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:04.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.898+0000 7fde6eb9d640 1 -- 172.21.15.87:0/3487516750 >> 172.21.15.87:0/3487516750 conn(0x7fde680d10c0 msgr2=0x7fde6809fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:04.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.898+0000 7fde6eb9d640 1 -- 172.21.15.87:0/3487516750 shutdown_connections 2024-09-17T03:15:04.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:04.898+0000 7fde6eb9d640 1 -- 172.21.15.87:0/3487516750 wait complete. 2024-09-17T03:15:05.310 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T03:14:55.675616+0000', 'modified': '2024-09-17T03:14:57.683823+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5004}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5004': {'gid': 5004, 'name': 'c', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.87:6834/1243349323', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6833', 'nonce': 1243349323}, {'type': 'v1', 'addr': '172.21.15.87:6834', 'nonce': 1243349323}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5004, 'qdb_cluster': [5004]} 2024-09-17T03:15:05.310 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T03:15:05.310 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-17T03:15:05.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.427+0000 7fc19f80e640 1 -- 172.21.15.87:0/2191701782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc184004800 msgr2=0x7fc184004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:05.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.427+0000 7fc19f80e640 1 --2- 172.21.15.87:0/2191701782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc184004800 0x7fc184004c10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc1980c5d00 tx=0x7fc184005830 comp rx=0 tx=0).stop 2024-09-17T03:15:05.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.427+0000 7fc19ddab640 1 -- 172.21.15.87:0/2191701782 reap_dead start 2024-09-17T03:15:05.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.427+0000 7fc19f80e640 1 -- 172.21.15.87:0/2191701782 shutdown_connections 2024-09-17T03:15:05.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.427+0000 7fc19f80e640 1 -- 172.21.15.87:0/2191701782 >> 172.21.15.87:0/2191701782 conn(0x7fc1980d2230 msgr2=0x7fc1980d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:05.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.427+0000 7fc19f80e640 1 -- 172.21.15.87:0/2191701782 shutdown_connections 2024-09-17T03:15:05.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.427+0000 7fc19f80e640 1 -- 172.21.15.87:0/2191701782 wait complete. 2024-09-17T03:15:05.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.429+0000 7fc19f80e640 1 Processor -- start 2024-09-17T03:15:05.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.429+0000 7fc19f80e640 1 -- start start 2024-09-17T03:15:05.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.430+0000 7fc19f80e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc184004800 0x7fc19809f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:05.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.430+0000 7fc19f80e640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc1981e1a10 0x7fc1981e1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:05.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.430+0000 7fc19f80e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc19809e030 0x7fc1981f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:05.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.430+0000 7fc19f80e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc1980b3040 con 0x7fc184004800 2024-09-17T03:15:05.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.430+0000 7fc19f80e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc1980b2ec0 con 0x7fc19809e030 2024-09-17T03:15:05.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.430+0000 7fc19f80e640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc1980d0f50 con 0x7fc1981e1a10 2024-09-17T03:15:05.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.430+0000 7fc19ddab640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc19809e030 0x7fc1981f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:05.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.430+0000 7fc19cda9640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc1981e1a10 0x7fc1981e1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:05.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.430+0000 7fc19ddab640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc19809e030 0x7fc1981f6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.87:34682/0 (socket says 172.21.15.87:34682) 2024-09-17T03:15:05.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.430+0000 7fc19ddab640 1 -- 172.21.15.87:0/2791290608 learned_addr learned my addr 172.21.15.87:0/2791290608 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:05.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.430+0000 7fc19d5aa640 1 --2- 172.21.15.87:0/2791290608 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc184004800 0x7fc19809f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:05.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.431+0000 7fc19cda9640 1 -- 172.21.15.87:0/2791290608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc19809e030 msgr2=0x7fc1981f6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:05.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.431+0000 7fc19cda9640 1 --2- 172.21.15.87:0/2791290608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc19809e030 0x7fc1981f6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.431+0000 7fc19cda9640 1 -- 172.21.15.87:0/2791290608 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc184004800 msgr2=0x7fc19809f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:05.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.431+0000 7fc19cda9640 1 --2- 172.21.15.87:0/2791290608 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc184004800 0x7fc19809f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.431+0000 7fc19cda9640 1 -- 172.21.15.87:0/2791290608 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1980cdc30 con 0x7fc1981e1a10 2024-09-17T03:15:05.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.431+0000 7fc19cda9640 1 --2- 172.21.15.87:0/2791290608 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc1981e1a10 0x7fc1981e1250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc188033000 tx=0x7fc188032670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:05.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.431+0000 7fc18dffb640 1 -- 172.21.15.87:0/2791290608 <== mon.2 v2:172.21.15.149:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188041070 con 0x7fc1981e1a10 2024-09-17T03:15:05.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.431+0000 7fc18dffb640 1 -- 172.21.15.87:0/2791290608 <== mon.2 v2:172.21.15.149:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc188032d30 con 0x7fc1981e1a10 2024-09-17T03:15:05.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.431+0000 7fc19f80e640 1 -- 172.21.15.87:0/2791290608 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1980cdf10 con 0x7fc1981e1a10 2024-09-17T03:15:05.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.431+0000 7fc18dffb640 1 -- 172.21.15.87:0/2791290608 <== mon.2 v2:172.21.15.149:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc188004290 con 0x7fc1981e1a10 2024-09-17T03:15:05.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.431+0000 7fc19f80e640 1 -- 172.21.15.87:0/2791290608 --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1981e1790 con 0x7fc1981e1a10 2024-09-17T03:15:05.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.433+0000 7fc18dffb640 1 -- 172.21.15.87:0/2791290608 <== mon.2 v2:172.21.15.149:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc1880035f0 con 0x7fc1981e1a10 2024-09-17T03:15:05.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.434+0000 7fc18dffb640 1 --2- 172.21.15.87:0/2791290608 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc17c078b90 0x7fc17c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:05.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.434+0000 7fc18dffb640 1 -- 172.21.15.87:0/2791290608 <== mon.2 v2:172.21.15.149:3301/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fc1880bb940 con 0x7fc1981e1a10 2024-09-17T03:15:05.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.434+0000 7fc19d5aa640 1 --2- 172.21.15.87:0/2791290608 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc17c078b90 0x7fc17c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:05.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.434+0000 7fc1a6858740 1 Processor -- start 2024-09-17T03:15:05.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.435+0000 7fc19d5aa640 1 --2- 172.21.15.87:0/2791290608 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc17c078b90 0x7fc17c07b0a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc194001240 tx=0x7fc194006720 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:05.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.435+0000 7fc1a6858740 1 -- start start 2024-09-17T03:15:05.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.436+0000 7fc1a6858740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b48270 0x558be5f19fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:05.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.436+0000 7fc1a6858740 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558be5b1f3b0 0x558be5f1a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:05.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.436+0000 7fc1a6858740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558be5b27590 0x558be5f373e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:05.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.436+0000 7fc1a6858740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x558be5e62820 con 0x558be5b48270 2024-09-17T03:15:05.444 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.436+0000 7fc1a6858740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558be5e51c50 con 0x558be5b27590 2024-09-17T03:15:05.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.436+0000 7fc1a6858740 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x558be5e55a60 con 0x558be5b1f3b0 2024-09-17T03:15:05.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.436+0000 7fc19d5aa640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b48270 0x558be5f19fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:05.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.436+0000 7fc19d5aa640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b48270 0x558be5f19fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34322/0 (socket says 172.21.15.87:34322) 2024-09-17T03:15:05.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.436+0000 7fc19d5aa640 1 -- 172.21.15.87:0/2865801575 learned_addr learned my addr 172.21.15.87:0/2865801575 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:05.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.436+0000 7fc19cda9640 1 --2- 172.21.15.87:0/2865801575 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558be5b1f3b0 0x558be5f1a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:05.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.436+0000 7fc19ddab640 1 --2- 172.21.15.87:0/2865801575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558be5b27590 0x558be5f373e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:05.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.436+0000 7fc19d5aa640 1 -- 172.21.15.87:0/2865801575 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558be5b1f3b0 msgr2=0x558be5f1a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:15:05.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.436+0000 7fc19d5aa640 1 --2- 172.21.15.87:0/2865801575 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558be5b1f3b0 0x558be5f1a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.436+0000 7fc19d5aa640 1 -- 172.21.15.87:0/2865801575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558be5b27590 msgr2=0x558be5f373e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:15:05.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.436+0000 7fc19d5aa640 1 --2- 172.21.15.87:0/2865801575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558be5b27590 0x558be5f373e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.436+0000 7fc19d5aa640 1 -- 172.21.15.87:0/2865801575 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be5bb3850 con 0x558be5b48270 2024-09-17T03:15:05.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.436+0000 7fc19ddab640 1 --2- 172.21.15.87:0/2865801575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558be5b27590 0x558be5f373e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:15:05.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.436+0000 7fc19d5aa640 1 --2- 172.21.15.87:0/2865801575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b48270 0x558be5f19fa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc1940066f0 tx=0x7fc194015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2255174577b9139 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:05.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.436+0000 7fc182ffd640 1 -- 172.21.15.87:0/2865801575 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194012a50 con 0x558be5b48270 2024-09-17T03:15:05.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.437+0000 7fc182ffd640 1 -- 172.21.15.87:0/2865801575 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc194024020 con 0x558be5b48270 2024-09-17T03:15:05.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.437+0000 7fc182ffd640 1 -- 172.21.15.87:0/2865801575 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194012c50 con 0x558be5b48270 2024-09-17T03:15:05.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.437+0000 7fc1a6858740 1 -- 172.21.15.87:0/2865801575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b48270 msgr2=0x558be5f19fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:05.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.437+0000 7fc1a6858740 1 --2- 172.21.15.87:0/2865801575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b48270 0x558be5f19fa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc1940066f0 tx=0x7fc194015000 comp rx=0 tx=0).stop 2024-09-17T03:15:05.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.437+0000 7fc1a6858740 1 -- 172.21.15.87:0/2865801575 shutdown_connections 2024-09-17T03:15:05.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.437+0000 7fc1a6858740 1 --2- 172.21.15.87:0/2865801575 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558be5b27590 0x558be5f373e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.437+0000 7fc1a6858740 1 --2- 172.21.15.87:0/2865801575 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558be5b1f3b0 0x558be5f1a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.437+0000 7fc1a6858740 1 --2- 172.21.15.87:0/2865801575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b48270 0x558be5f19fa0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.437+0000 7fc1a6858740 1 -- 172.21.15.87:0/2865801575 >> 172.21.15.87:0/2865801575 conn(0x558be5b3c3c0 msgr2=0x558be5ef0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:05.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.437+0000 7fc1a6858740 1 -- 172.21.15.87:0/2865801575 shutdown_connections 2024-09-17T03:15:05.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.437+0000 7fc1a6858740 1 -- 172.21.15.87:0/2865801575 wait complete. 2024-09-17T03:15:05.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.438+0000 7fc1a6858740 1 -- start start 2024-09-17T03:15:05.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.438+0000 7fc1a6858740 1 Processor -- start 2024-09-17T03:15:05.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.440+0000 7fc1a6858740 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558be5b1f3b0 0x558be5f45380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:05.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.440+0000 7fc1a6858740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558be5b27590 0x558be5ee96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:05.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.440+0000 7fc1a6858740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b48270 0x558be5f37140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:05.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.440+0000 7fc1a6858740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x558be5bc2620 con 0x558be5b48270 2024-09-17T03:15:05.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.440+0000 7fc1a6858740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558be5df75f0 con 0x558be5b27590 2024-09-17T03:15:05.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.440+0000 7fc1a6858740 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x558be5f1aa20 con 0x558be5b1f3b0 2024-09-17T03:15:05.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.440+0000 7fc1a6858740 10 client.0 _ll_register_callbacks cb 0x558be5dfb0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:15:05.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.440+0000 7fc19ddab640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b48270 0x558be5f37140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:05.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.440+0000 7fc19ddab640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b48270 0x558be5f37140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34330/0 (socket says 172.21.15.87:34330) 2024-09-17T03:15:05.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.440+0000 7fc19ddab640 1 -- 172.21.15.87:0/2438835815 learned_addr learned my addr 172.21.15.87:0/2438835815 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:05.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.440+0000 7fc19d5aa640 1 --2- 172.21.15.87:0/2438835815 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558be5b1f3b0 0x558be5f45380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:05.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.440+0000 7fc19cda9640 1 --2- 172.21.15.87:0/2438835815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558be5b27590 0x558be5ee96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:05.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.440+0000 7fc19ddab640 1 -- 172.21.15.87:0/2438835815 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558be5b1f3b0 msgr2=0x558be5f45380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:15:05.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.440+0000 7fc19ddab640 1 --2- 172.21.15.87:0/2438835815 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558be5b1f3b0 0x558be5f45380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.440+0000 7fc19ddab640 1 -- 172.21.15.87:0/2438835815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558be5b27590 msgr2=0x558be5ee96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:15:05.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.440+0000 7fc19ddab640 1 --2- 172.21.15.87:0/2438835815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558be5b27590 0x558be5ee96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.440+0000 7fc19ddab640 1 -- 172.21.15.87:0/2438835815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be5c2a6b0 con 0x558be5b48270 2024-09-17T03:15:05.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.440+0000 7fc19ddab640 1 --2- 172.21.15.87:0/2438835815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b48270 0x558be5f37140 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc1900068c0 tx=0x7fc190001b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4705251629aea75 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:05.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.440+0000 7fc177fff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-09-17T03:15:05.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.440+0000 7fc177fff640 1 -- 172.21.15.87:0/2438835815 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc190011840 con 0x558be5b48270 2024-09-17T03:15:05.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.440+0000 7fc177fff640 1 -- 172.21.15.87:0/2438835815 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fc1900119e0 con 0x558be5b48270 2024-09-17T03:15:05.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.440+0000 7fc1a6858740 1 -- 172.21.15.87:0/2438835815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc194024020 con 0x558be5b48270 2024-09-17T03:15:05.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.441+0000 7fc177fff640 1 -- 172.21.15.87:0/2438835815 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) ==== 24+0+0 (secure 0 0 0) 0x7fc190020050 con 0x558be5b48270 2024-09-17T03:15:05.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.441+0000 7fc1a6858740 10 client.5199 fetch_fsmap learned FSMap version 13 2024-09-17T03:15:05.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.441+0000 7fc1a6858740 1 -- 172.21.15.87:0/2438835815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) -- 0x558be5f38000 con 0x558be5b48270 2024-09-17T03:15:05.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.441+0000 7fc177fff640 1 -- 172.21.15.87:0/2438835815 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc194024020 con 0x558be5b48270 2024-09-17T03:15:05.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.441+0000 7fc177fff640 1 -- 172.21.15.87:0/2438835815 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 13) ==== 3502+0+0 (secure 0 0 0) 0x7fc190023020 con 0x558be5b48270 2024-09-17T03:15:05.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.441+0000 7fc1a6858740 10 client.5199 fetch_fsmap finished waiting for FSMap version 13 2024-09-17T03:15:05.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.441+0000 7fc1a6858740 10 client.5199 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T03:15:05.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.441+0000 7fc1a6858740 20 client.5199 populate_metadata read hostname 'smithi087' 2024-09-17T03:15:05.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.441+0000 7fc1a6858740 1 --2- 172.21.15.87:0/2438835815 >> [v2:172.21.15.87:6833/1243349323,v1:172.21.15.87:6834/1243349323] conn(0x558be5b06e40 0x558be5f477b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:05.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.441+0000 7fc1a6858740 4 client.5199 mds_command: new command op to 5004 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T03:15:05.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.441+0000 7fc1a6858740 1 -- 172.21.15.87:0/2438835815 --> [v2:172.21.15.87:6833/1243349323,v1:172.21.15.87:6834/1243349323] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558be5e64770 con 0x558be5b06e40 2024-09-17T03:15:05.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.441+0000 7fc19d5aa640 1 --2- 172.21.15.87:0/2438835815 >> [v2:172.21.15.87:6833/1243349323,v1:172.21.15.87:6834/1243349323] conn(0x558be5b06e40 0x558be5f477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:05.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.441+0000 7fc177fff640 10 client.5199 ms_handle_connect on v2:172.21.15.87:6833/1243349323 2024-09-17T03:15:05.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.441+0000 7fc19d5aa640 1 --2- 172.21.15.87:0/2438835815 >> [v2:172.21.15.87:6833/1243349323,v1:172.21.15.87:6834/1243349323] conn(0x558be5b06e40 0x558be5f477b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d56588a8244ae3f1 server_cookie=e3861f70e6df6389 in_seq=0 out_seq=0 2024-09-17T03:15:05.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.442+0000 7fc177fff640 1 -- 172.21.15.87:0/2438835815 <== mds.0 v2:172.21.15.87:6833/1243349323 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558be5e64770 con 0x558be5b06e40 2024-09-17T03:15:05.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.442+0000 7fc177fff640 10 client.5199 handle_command_reply: tid=0 2024-09-17T03:15:05.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.442+0000 7fc1a6858740 1 client.5199 shutdown 2024-09-17T03:15:05.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.442+0000 7fc1a6858740 1 -- 172.21.15.87:0/2438835815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b48270 msgr2=0x558be5f37140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:05.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.442+0000 7fc1a6858740 1 --2- 172.21.15.87:0/2438835815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b48270 0x558be5f37140 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc1900068c0 tx=0x7fc190001b70 comp rx=0 tx=0).stop 2024-09-17T03:15:05.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.442+0000 7fc1a6858740 1 -- 172.21.15.87:0/2438835815 shutdown_connections 2024-09-17T03:15:05.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.442+0000 7fc1a6858740 1 --2- 172.21.15.87:0/2438835815 >> [v2:172.21.15.87:6833/1243349323,v1:172.21.15.87:6834/1243349323] conn(0x558be5b06e40 0x558be5f477b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.442+0000 7fc1a6858740 1 --2- 172.21.15.87:0/2438835815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b48270 0x558be5f37140 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.442+0000 7fc177fff640 0 client.5199 ms_handle_reset on v2:172.21.15.87:6833/1243349323 2024-09-17T03:15:05.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.442+0000 7fc1a6858740 1 --2- 172.21.15.87:0/2438835815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558be5b27590 0x558be5ee96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.442+0000 7fc1a6858740 1 --2- 172.21.15.87:0/2438835815 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558be5b1f3b0 0x558be5f45380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.442+0000 7fc1a6858740 1 -- 172.21.15.87:0/2438835815 >> 172.21.15.87:0/2438835815 conn(0x558be5b3c3c0 msgr2=0x558be5f369b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:05.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.443+0000 7fc1a6858740 1 -- 172.21.15.87:0/2438835815 shutdown_connections 2024-09-17T03:15:05.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.443+0000 7fc1a6858740 1 -- 172.21.15.87:0/2438835815 wait complete. 2024-09-17T03:15:05.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.443+0000 7fc1a6858740 20 client.5199 trim_cache size 0 max 16384 2024-09-17T03:15:05.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.461+0000 7fc1a6858740 1 Processor -- start 2024-09-17T03:15:05.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.461+0000 7fc1a6858740 1 -- start start 2024-09-17T03:15:05.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.462+0000 7fc1a6858740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b06e40 0x558be5f084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:05.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.462+0000 7fc1a6858740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558be5b1f3b0 0x558be5f089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:05.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.462+0000 7fc1a6858740 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558be5b27590 0x558be5f57c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:05.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.462+0000 7fc1a6858740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc1981ff500 con 0x558be5b06e40 2024-09-17T03:15:05.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.462+0000 7fc1a6858740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc1981f3f80 con 0x558be5b1f3b0 2024-09-17T03:15:05.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.462+0000 7fc1a6858740 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc1981feeb0 con 0x558be5b27590 2024-09-17T03:15:05.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.462+0000 7fc19d5aa640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b06e40 0x558be5f084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:05.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.462+0000 7fc19d5aa640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b06e40 0x558be5f084a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34342/0 (socket says 172.21.15.87:34342) 2024-09-17T03:15:05.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.462+0000 7fc19d5aa640 1 -- 172.21.15.87:0/3236355545 learned_addr learned my addr 172.21.15.87:0/3236355545 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:05.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.462+0000 7fc19ddab640 1 --2- 172.21.15.87:0/3236355545 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558be5b27590 0x558be5f57c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:05.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.462+0000 7fc19cda9640 1 --2- 172.21.15.87:0/3236355545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558be5b1f3b0 0x558be5f089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:05.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.462+0000 7fc19d5aa640 1 -- 172.21.15.87:0/3236355545 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558be5b27590 msgr2=0x558be5f57c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:15:05.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.462+0000 7fc19d5aa640 1 --2- 172.21.15.87:0/3236355545 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558be5b27590 0x558be5f57c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.462+0000 7fc19d5aa640 1 -- 172.21.15.87:0/3236355545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558be5b1f3b0 msgr2=0x558be5f089e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:15:05.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.462+0000 7fc19d5aa640 1 --2- 172.21.15.87:0/3236355545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558be5b1f3b0 0x558be5f089e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.462+0000 7fc19d5aa640 1 -- 172.21.15.87:0/3236355545 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558be5f46fc0 con 0x558be5b06e40 2024-09-17T03:15:05.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.462+0000 7fc19ddab640 1 --2- 172.21.15.87:0/3236355545 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558be5b27590 0x558be5f57c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:15:05.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.462+0000 7fc19cda9640 1 --2- 172.21.15.87:0/3236355545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558be5b1f3b0 0x558be5f089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:15:05.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.463+0000 7fc19d5aa640 1 --2- 172.21.15.87:0/3236355545 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b06e40 0x558be5f084a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc194022040 tx=0x7fc1980b36d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c7dbc8d53d5aac0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:05.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.463+0000 7fc1827fc640 1 -- 172.21.15.87:0/3236355545 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194023040 con 0x558be5b06e40 2024-09-17T03:15:05.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.463+0000 7fc1827fc640 1 -- 172.21.15.87:0/3236355545 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc194035020 con 0x558be5b06e40 2024-09-17T03:15:05.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.463+0000 7fc1827fc640 1 -- 172.21.15.87:0/3236355545 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc194029a10 con 0x558be5b06e40 2024-09-17T03:15:05.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.463+0000 7fc1a6858740 1 -- 172.21.15.87:0/3236355545 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b06e40 msgr2=0x558be5f084a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:05.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.463+0000 7fc1a6858740 1 --2- 172.21.15.87:0/3236355545 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b06e40 0x558be5f084a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc194022040 tx=0x7fc1980b36d0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.463+0000 7fc1a6858740 1 -- 172.21.15.87:0/3236355545 shutdown_connections 2024-09-17T03:15:05.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.463+0000 7fc1a6858740 1 --2- 172.21.15.87:0/3236355545 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558be5b27590 0x558be5f57c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.463+0000 7fc1a6858740 1 --2- 172.21.15.87:0/3236355545 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558be5b1f3b0 0x558be5f089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.463+0000 7fc1a6858740 1 --2- 172.21.15.87:0/3236355545 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b06e40 0x558be5f084a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.463+0000 7fc1a6858740 1 -- 172.21.15.87:0/3236355545 >> 172.21.15.87:0/3236355545 conn(0x558be5b3c3c0 msgr2=0x558be5f11730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:05.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.463+0000 7fc1a6858740 1 -- 172.21.15.87:0/3236355545 shutdown_connections 2024-09-17T03:15:05.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.463+0000 7fc1a6858740 1 -- 172.21.15.87:0/3236355545 wait complete. 2024-09-17T03:15:05.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.464+0000 7fc1a6858740 1 -- start start 2024-09-17T03:15:05.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.464+0000 7fc1a6858740 1 Processor -- start 2024-09-17T03:15:05.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.466+0000 7fc1a6858740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b06e40 0x558be5fb17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:05.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.466+0000 7fc1a6858740 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558be5b1f3b0 0x558be5f4d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:05.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.466+0000 7fc1a6858740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558be5b27590 0x558be5f4dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:05.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.466+0000 7fc1a6858740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x558be5f37770 con 0x558be5b06e40 2024-09-17T03:15:05.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.466+0000 7fc1a6858740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x558be5f2d7f0 con 0x558be5b27590 2024-09-17T03:15:05.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.466+0000 7fc1a6858740 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x558be5f2e7b0 con 0x558be5b1f3b0 2024-09-17T03:15:05.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.466+0000 7fc1a6858740 10 client.0 _ll_register_callbacks cb 0x7fc190004cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:15:05.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.466+0000 7fc19d5aa640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b06e40 0x558be5fb17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:05.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.466+0000 7fc19d5aa640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b06e40 0x558be5fb17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34346/0 (socket says 172.21.15.87:34346) 2024-09-17T03:15:05.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.466+0000 7fc19d5aa640 1 -- 172.21.15.87:0/4259163438 learned_addr learned my addr 172.21.15.87:0/4259163438 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:05.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.466+0000 7fc19cda9640 1 --2- 172.21.15.87:0/4259163438 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558be5b1f3b0 0x558be5f4d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:05.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.466+0000 7fc19ddab640 1 --2- 172.21.15.87:0/4259163438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558be5b27590 0x558be5f4dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:05.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.466+0000 7fc19d5aa640 1 -- 172.21.15.87:0/4259163438 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558be5b1f3b0 msgr2=0x558be5f4d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:15:05.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.466+0000 7fc19d5aa640 1 --2- 172.21.15.87:0/4259163438 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558be5b1f3b0 0x558be5f4d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.466+0000 7fc19d5aa640 1 -- 172.21.15.87:0/4259163438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558be5b27590 msgr2=0x558be5f4dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:15:05.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.466+0000 7fc19d5aa640 1 --2- 172.21.15.87:0/4259163438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558be5b27590 0x558be5f4dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.466+0000 7fc19d5aa640 1 -- 172.21.15.87:0/4259163438 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558be5f452e0 con 0x558be5b06e40 2024-09-17T03:15:05.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.466+0000 7fc19cda9640 1 --2- 172.21.15.87:0/4259163438 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558be5b1f3b0 0x558be5f4d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:05.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.466+0000 7fc19ddab640 1 --2- 172.21.15.87:0/4259163438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558be5b27590 0x558be5f4dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:05.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.466+0000 7fc19d5aa640 1 --2- 172.21.15.87:0/4259163438 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b06e40 0x558be5fb17b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x558be5ef1e30 tx=0x7fc194006c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63eb503c4c159414 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:05.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.466+0000 7fc177fff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-09-17T03:15:05.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.467+0000 7fc177fff640 1 -- 172.21.15.87:0/4259163438 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19403e020 con 0x558be5b06e40 2024-09-17T03:15:05.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.467+0000 7fc177fff640 1 -- 172.21.15.87:0/4259163438 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fc194029850 con 0x558be5b06e40 2024-09-17T03:15:05.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.467+0000 7fc1a6858740 1 -- 172.21.15.87:0/4259163438 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc194035020 con 0x558be5b06e40 2024-09-17T03:15:05.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.467+0000 7fc177fff640 1 -- 172.21.15.87:0/4259163438 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) ==== 24+0+0 (secure 0 0 0) 0x7fc194048050 con 0x558be5b06e40 2024-09-17T03:15:05.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.467+0000 7fc1a6858740 10 client.5205 fetch_fsmap learned FSMap version 13 2024-09-17T03:15:05.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.467+0000 7fc1a6858740 1 -- 172.21.15.87:0/4259163438 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) -- 0x558be5ef1900 con 0x558be5b06e40 2024-09-17T03:15:05.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.467+0000 7fc177fff640 1 -- 172.21.15.87:0/4259163438 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc194035020 con 0x558be5b06e40 2024-09-17T03:15:05.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.467+0000 7fc177fff640 1 -- 172.21.15.87:0/4259163438 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 13) ==== 3502+0+0 (secure 0 0 0) 0x7fc19404b020 con 0x558be5b06e40 2024-09-17T03:15:05.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.467+0000 7fc1a6858740 10 client.5205 fetch_fsmap finished waiting for FSMap version 13 2024-09-17T03:15:05.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.467+0000 7fc1a6858740 10 client.5205 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-17T03:15:05.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.467+0000 7fc1a6858740 20 client.5205 populate_metadata read hostname 'smithi087' 2024-09-17T03:15:05.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.467+0000 7fc1a6858740 1 --2- 172.21.15.87:0/4259163438 >> [v2:172.21.15.87:6833/1243349323,v1:172.21.15.87:6834/1243349323] conn(0x558be5adc280 0x558be5fb4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:05.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.467+0000 7fc1a6858740 4 client.5205 mds_command: new command op to 5004 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T03:15:05.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.467+0000 7fc1a6858740 1 -- 172.21.15.87:0/4259163438 --> [v2:172.21.15.87:6833/1243349323,v1:172.21.15.87:6834/1243349323] -- command(tid 0: {"prefix": "status"}) -- 0x558be5e60220 con 0x558be5adc280 2024-09-17T03:15:05.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.467+0000 7fc19cda9640 1 --2- 172.21.15.87:0/4259163438 >> [v2:172.21.15.87:6833/1243349323,v1:172.21.15.87:6834/1243349323] conn(0x558be5adc280 0x558be5fb4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:05.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.467+0000 7fc177fff640 10 client.5205 ms_handle_connect on v2:172.21.15.87:6833/1243349323 2024-09-17T03:15:05.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.468+0000 7fc19cda9640 1 --2- 172.21.15.87:0/4259163438 >> [v2:172.21.15.87:6833/1243349323,v1:172.21.15.87:6834/1243349323] conn(0x558be5adc280 0x558be5fb4440 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cc2322a0fcf5379 server_cookie=a31907997422735c in_seq=0 out_seq=0 2024-09-17T03:15:05.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.468+0000 7fc177fff640 1 -- 172.21.15.87:0/4259163438 <== mds.0 v2:172.21.15.87:6833/1243349323 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x558be5e60220 con 0x558be5adc280 2024-09-17T03:15:05.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.468+0000 7fc177fff640 10 client.5205 handle_command_reply: tid=0 2024-09-17T03:15:05.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.468+0000 7fc1a6858740 1 client.5205 shutdown 2024-09-17T03:15:05.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.468+0000 7fc1a6858740 1 -- 172.21.15.87:0/4259163438 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b06e40 msgr2=0x558be5fb17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:05.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.468+0000 7fc1a6858740 1 --2- 172.21.15.87:0/4259163438 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b06e40 0x558be5fb17b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x558be5ef1e30 tx=0x7fc194006c50 comp rx=0 tx=0).stop 2024-09-17T03:15:05.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.468+0000 7fc1a6858740 1 -- 172.21.15.87:0/4259163438 shutdown_connections 2024-09-17T03:15:05.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.468+0000 7fc1a6858740 1 --2- 172.21.15.87:0/4259163438 >> [v2:172.21.15.87:6833/1243349323,v1:172.21.15.87:6834/1243349323] conn(0x558be5adc280 0x558be5fb4440 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.468+0000 7fc1a6858740 1 --2- 172.21.15.87:0/4259163438 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x558be5b27590 0x558be5f4dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.468+0000 7fc1a6858740 1 --2- 172.21.15.87:0/4259163438 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x558be5b1f3b0 0x558be5f4d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.468+0000 7fc177fff640 0 client.5205 ms_handle_reset on v2:172.21.15.87:6833/1243349323 2024-09-17T03:15:05.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.468+0000 7fc1a6858740 1 --2- 172.21.15.87:0/4259163438 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558be5b06e40 0x558be5fb17b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.468+0000 7fc1a6858740 1 -- 172.21.15.87:0/4259163438 >> 172.21.15.87:0/4259163438 conn(0x558be5b3c3c0 msgr2=0x558be5f2c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:05.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.468+0000 7fc1a6858740 1 -- 172.21.15.87:0/4259163438 shutdown_connections 2024-09-17T03:15:05.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.468+0000 7fc1a6858740 1 -- 172.21.15.87:0/4259163438 wait complete. 2024-09-17T03:15:05.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.468+0000 7fc1a6858740 20 client.5205 trim_cache size 0 max 16384 2024-09-17T03:15:05.475 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-17T03:15:05.475 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "7b2553aa-e562-4da8-b175-8e91f460da55", 2024-09-17T03:15:05.475 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-09-17T03:15:05.475 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 5004, 2024-09-17T03:15:05.475 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-09-17T03:15:05.475 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-09-17T03:15:05.476 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-09-17T03:15:05.476 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 9.1468615310000008, 2024-09-17T03:15:05.476 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 13, 2024-09-17T03:15:05.476 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 34, 2024-09-17T03:15:05.476 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 34, 2024-09-17T03:15:05.476 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 16.169561145999999 2024-09-17T03:15:05.476 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-17T03:15:05.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.469+0000 7fc19f80e640 1 -- 172.21.15.87:0/2791290608 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc17c078b90 msgr2=0x7fc17c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:05.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.469+0000 7fc19f80e640 1 --2- 172.21.15.87:0/2791290608 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc17c078b90 0x7fc17c07b0a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc194001240 tx=0x7fc194006720 comp rx=0 tx=0).stop 2024-09-17T03:15:05.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.469+0000 7fc19f80e640 1 -- 172.21.15.87:0/2791290608 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc1981e1a10 msgr2=0x7fc1981e1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:05.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.469+0000 7fc19f80e640 1 --2- 172.21.15.87:0/2791290608 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc1981e1a10 0x7fc1981e1250 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc188033000 tx=0x7fc188032670 comp rx=0 tx=0).stop 2024-09-17T03:15:05.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.469+0000 7fc19f80e640 1 -- 172.21.15.87:0/2791290608 shutdown_connections 2024-09-17T03:15:05.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.469+0000 7fc19f80e640 1 --2- 172.21.15.87:0/2791290608 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc17c078b90 0x7fc17c07b0a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.469+0000 7fc19f80e640 1 --2- 172.21.15.87:0/2791290608 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc19809e030 0x7fc1981f6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.469+0000 7fc19f80e640 1 --2- 172.21.15.87:0/2791290608 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc1981e1a10 0x7fc1981e1250 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.469+0000 7fc19f80e640 1 --2- 172.21.15.87:0/2791290608 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc184004800 0x7fc19809f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.469+0000 7fc19f80e640 1 -- 172.21.15.87:0/2791290608 >> 172.21.15.87:0/2791290608 conn(0x7fc1980d2230 msgr2=0x7fc1981e5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:05.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.469+0000 7fc19f80e640 1 -- 172.21.15.87:0/2791290608 shutdown_connections 2024-09-17T03:15:05.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.469+0000 7fc19f80e640 1 -- 172.21.15.87:0/2791290608 wait complete. 2024-09-17T03:15:05.479 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T03:15:05.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.593+0000 7fca2d732640 1 -- 172.21.15.87:0/1615246986 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca14003a50 msgr2=0x7fca14003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:05.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.593+0000 7fca2d732640 1 --2- 172.21.15.87:0/1615246986 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca14003a50 0x7fca14003ec0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fca1800bb20 tx=0x7fca14005830 comp rx=0 tx=0).stop 2024-09-17T03:15:05.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.593+0000 7fca2d732640 1 -- 172.21.15.87:0/1615246986 shutdown_connections 2024-09-17T03:15:05.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.593+0000 7fca2d732640 1 --2- 172.21.15.87:0/1615246986 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca14003a50 0x7fca14003ec0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.593+0000 7fca2d732640 1 -- 172.21.15.87:0/1615246986 >> 172.21.15.87:0/1615246986 conn(0x7fca2808dae0 msgr2=0x7fca2809d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:05.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.593+0000 7fca2d732640 1 -- 172.21.15.87:0/1615246986 shutdown_connections 2024-09-17T03:15:05.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.593+0000 7fca2d732640 1 -- 172.21.15.87:0/1615246986 wait complete. 2024-09-17T03:15:05.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.595+0000 7fca2d732640 1 Processor -- start 2024-09-17T03:15:05.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.595+0000 7fca2d732640 1 -- start start 2024-09-17T03:15:05.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.596+0000 7fca2d732640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca14003a50 0x7fca281f8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:05.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.596+0000 7fca2d732640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca2815b840 0x7fca28158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:05.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.596+0000 7fca2d732640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fca2806ef10 0x7fca2806f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:05.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.596+0000 7fca2d732640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fca280d33f0 con 0x7fca14003a50 2024-09-17T03:15:05.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.596+0000 7fca2d732640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fca280d3270 con 0x7fca2815b840 2024-09-17T03:15:05.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.596+0000 7fca2d732640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fca280d4560 con 0x7fca2806ef10 2024-09-17T03:15:05.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.596+0000 7fca26ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca14003a50 0x7fca281f8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:05.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.596+0000 7fca26ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca14003a50 0x7fca281f8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34370/0 (socket says 172.21.15.87:34370) 2024-09-17T03:15:05.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.596+0000 7fca26ffd640 1 -- 172.21.15.87:0/2033680968 learned_addr learned my addr 172.21.15.87:0/2033680968 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:05.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.597+0000 7fca277fe640 1 --2- 172.21.15.87:0/2033680968 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fca2806ef10 0x7fca2806f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:05.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.597+0000 7fca1e7fc640 1 --2- 172.21.15.87:0/2033680968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca2815b840 0x7fca28158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:05.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.597+0000 7fca26ffd640 1 -- 172.21.15.87:0/2033680968 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fca2806ef10 msgr2=0x7fca2806f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:05.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.597+0000 7fca26ffd640 1 --2- 172.21.15.87:0/2033680968 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fca2806ef10 0x7fca2806f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.597+0000 7fca26ffd640 1 -- 172.21.15.87:0/2033680968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca2815b840 msgr2=0x7fca28158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:05.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.597+0000 7fca26ffd640 1 --2- 172.21.15.87:0/2033680968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca2815b840 0x7fca28158450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.597+0000 7fca26ffd640 1 -- 172.21.15.87:0/2033680968 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2806f860 con 0x7fca14003a50 2024-09-17T03:15:05.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.597+0000 7fca1e7fc640 1 --2- 172.21.15.87:0/2033680968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca2815b840 0x7fca28158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:05.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.597+0000 7fca26ffd640 1 --2- 172.21.15.87:0/2033680968 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca14003a50 0x7fca281f8e10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fca1800ba40 tx=0x7fca1800b770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:05.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.597+0000 7fca1ffff640 1 -- 172.21.15.87:0/2033680968 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca18040070 con 0x7fca14003a50 2024-09-17T03:15:05.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.597+0000 7fca1ffff640 1 -- 172.21.15.87:0/2033680968 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca18004340 con 0x7fca14003a50 2024-09-17T03:15:05.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.597+0000 7fca2d732640 1 -- 172.21.15.87:0/2033680968 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca2815c1f0 con 0x7fca14003a50 2024-09-17T03:15:05.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.597+0000 7fca1ffff640 1 -- 172.21.15.87:0/2033680968 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca18035040 con 0x7fca14003a50 2024-09-17T03:15:05.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.597+0000 7fca2d732640 1 -- 172.21.15.87:0/2033680968 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca281f9940 con 0x7fca14003a50 2024-09-17T03:15:05.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.598+0000 7fca2d732640 1 -- 172.21.15.87:0/2033680968 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca280d4ac0 con 0x7fca14003a50 2024-09-17T03:15:05.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.599+0000 7fca1ffff640 1 -- 172.21.15.87:0/2033680968 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fca18038020 con 0x7fca14003a50 2024-09-17T03:15:05.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.599+0000 7fca1ffff640 1 --2- 172.21.15.87:0/2033680968 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fca08078b90 0x7fca0807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:05.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.600+0000 7fca1ffff640 1 -- 172.21.15.87:0/2033680968 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fca180bba20 con 0x7fca14003a50 2024-09-17T03:15:05.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.600+0000 7fca1e7fc640 1 --2- 172.21.15.87:0/2033680968 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fca08078b90 0x7fca0807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:05.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.600+0000 7fca1e7fc640 1 --2- 172.21.15.87:0/2033680968 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fca08078b90 0x7fca0807b0a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fca100018a0 tx=0x7fca10001db0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:05.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.604+0000 7fca1ffff640 1 -- 172.21.15.87:0/2033680968 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fca2806f860 con 0x7fca14003a50 2024-09-17T03:15:05.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.813+0000 7fca2d732640 1 -- 172.21.15.87:0/2033680968 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fca2806de10 con 0x7fca14003a50 2024-09-17T03:15:05.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.814+0000 7fca1ffff640 1 -- 172.21.15.87:0/2033680968 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) ==== 53+0+73 (secure 0 0 0) 0x7fca18083e60 con 0x7fca14003a50 2024-09-17T03:15:05.813 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T03:15:05.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.816+0000 7fca2d732640 1 -- 172.21.15.87:0/2033680968 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fca08078b90 msgr2=0x7fca0807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:05.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.816+0000 7fca2d732640 1 --2- 172.21.15.87:0/2033680968 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fca08078b90 0x7fca0807b0a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fca100018a0 tx=0x7fca10001db0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.816+0000 7fca2d732640 1 -- 172.21.15.87:0/2033680968 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca14003a50 msgr2=0x7fca281f8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:05.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.816+0000 7fca2d732640 1 --2- 172.21.15.87:0/2033680968 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca14003a50 0x7fca281f8e10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fca1800ba40 tx=0x7fca1800b770 comp rx=0 tx=0).stop 2024-09-17T03:15:05.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.816+0000 7fca2d732640 1 -- 172.21.15.87:0/2033680968 shutdown_connections 2024-09-17T03:15:05.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.816+0000 7fca2d732640 1 --2- 172.21.15.87:0/2033680968 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fca08078b90 0x7fca0807b0a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.816+0000 7fca2d732640 1 --2- 172.21.15.87:0/2033680968 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fca2806ef10 0x7fca2806f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.816+0000 7fca2d732640 1 --2- 172.21.15.87:0/2033680968 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fca2815b840 0x7fca28158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.816+0000 7fca2d732640 1 --2- 172.21.15.87:0/2033680968 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca14003a50 0x7fca281f8e10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:05.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.816+0000 7fca2d732640 1 -- 172.21.15.87:0/2033680968 >> 172.21.15.87:0/2033680968 conn(0x7fca2808dae0 msgr2=0x7fca28161690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:05.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.816+0000 7fca2d732640 1 -- 172.21.15.87:0/2033680968 shutdown_connections 2024-09-17T03:15:05.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:05.816+0000 7fca2d732640 1 -- 172.21.15.87:0/2033680968 wait complete. 2024-09-17T03:15:05.826 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T03:15:05.826 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T03:15:05.826 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-09-17T03:15:05.826 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T03:15:05.826 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:05.826 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T03:15:05.826 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T03:15:05.826 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T03:15:05.826 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:15:05.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:05.827 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-09-17T03:15:05.843 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:15:05.843 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:15:05.843 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:15:05.843 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:05.844 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-09-17T03:15:05.844 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:05.844 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:15:05.844 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-09-17T03:15:05.844 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-09-17T03:15:05.845 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:15:05.845 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-09-17T03:15:05.845 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-09-17T03:15:05.845 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:15:05.845 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:15:05.846 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T03:15:05.846 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:15:05.846 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T03:15:05.846 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42726sec preferred_lft 42726sec 2024-09-17T03:15:05.846 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link noprefixroute 2024-09-17T03:15:05.846 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:05.847 INFO:teuthology.orchestra.run.smithi087.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:15:05.847 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 26:32:f7:da:05:b7 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:15:05.847 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T03:15:05.847 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:05.847 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c822:ceff:fef0:4a99/64 scope link 2024-09-17T03:15:05.847 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:05.848 INFO:teuthology.orchestra.run.smithi087.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:15:05.848 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether b2:cb:c1:94:14:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:15:05.848 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::b0cb:c1ff:fe94:145d/64 scope link 2024-09-17T03:15:05.848 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:05.848 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:15:05.849 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 26:32:f7:da:05:b7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:15:05.849 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::2432:f7ff:feda:5b7/64 scope link 2024-09-17T03:15:05.849 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:05.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:05.850 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T03:15:05.907 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-09-17T03:15:05.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:05.908 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-09-17T03:15:05.965 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-09-17T03:15:05.965 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T03:15:05.965 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T03:15:05.965 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T03:15:05.965 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T03:15:05.965 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:15:05.966 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-09-17T03:15:05.966 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T03:15:05.966 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T03:15:05.966 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T03:15:05.966 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-09-17T03:15:06.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:15:06.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:15:06.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:06.092 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-09-17T03:15:06.108 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T03:15:06.108 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T03:15:06.109 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:15:06.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:06.109 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-09-17T03:15:06.174 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T03:15:06.174 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:15:06.174 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:06.229 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:15:06.229 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:15:06.229 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:06.284 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-09-17T03:15:06.349 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T03:15:06.406 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-09-17T03:15:06.406 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-09-17T03:15:06.406 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-09-17T03:15:06.406 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-09-17T03:15:06.406 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-09-17T03:15:06.406 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-09-17T03:15:06.406 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-09-17T03:15:06.406 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-09-17T03:15:06.406 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-09-17T03:15:06.406 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-09-17T03:15:06.406 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-09-17T03:15:06.407 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-09-17T03:15:06.407 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T03:15:06.407 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-09-17T03:15:06.407 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-09-17T03:15:06.407 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-09-17T03:15:06.407 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-09-17T03:15:06.407 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-09-17T03:15:06.407 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-09-17T03:15:06.407 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T03:15:06.407 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T03:15:06.407 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T03:15:06.407 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T03:15:06.407 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-09-17T03:15:06.408 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-09-17T03:15:06.408 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T03:15:06.408 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T03:15:06.408 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T03:15:06.408 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T03:15:06.408 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-09-17T03:15:06.408 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:15:06.408 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:15:06.408 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:15:06.408 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:15:06.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:06.409 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-09-17T03:15:06.462 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T03:15:06.462 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-17T03:15:06.504 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-09-17T03:15:06.528 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T03:15:06.587 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-09-17T03:15:06.587 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-09-17T03:15:06.587 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-09-17T03:15:06.587 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-09-17T03:15:06.587 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-09-17T03:15:06.587 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-09-17T03:15:06.587 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-09-17T03:15:06.587 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-09-17T03:15:06.588 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-09-17T03:15:06.588 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-09-17T03:15:06.588 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-09-17T03:15:06.588 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-09-17T03:15:06.588 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T03:15:06.588 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-09-17T03:15:06.588 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-09-17T03:15:06.588 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-09-17T03:15:06.588 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-09-17T03:15:06.588 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-09-17T03:15:06.588 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-09-17T03:15:06.588 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T03:15:06.588 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T03:15:06.589 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T03:15:06.589 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T03:15:06.589 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-09-17T03:15:06.589 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-09-17T03:15:06.589 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T03:15:06.589 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T03:15:06.589 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T03:15:06.589 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T03:15:06.589 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-09-17T03:15:06.589 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:15:06.589 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:15:06.589 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:15:06.589 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:15:06.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:06.590 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-09-17T03:15:06.597 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:2024-09-17T03:15:06.599+0000 7f03e13fe480 -1 init, newargv = 0x7f03c4004580 newargc=15 2024-09-17T03:15:06.598 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:ceph-fuse[49506]: starting ceph client 2024-09-17T03:15:06.603 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:ceph-fuse[49506]: starting fuse 2024-09-17T03:15:06.613 INFO:teuthology.orchestra.run.smithi087.stdout:42 2024-09-17T03:15:06.613 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-17T03:15:06.613 DEBUG:teuthology.orchestra.run.smithi087:> sudo stdin-killer -- python3 -c ' 2024-09-17T03:15:06.613 DEBUG:teuthology.orchestra.run.smithi087:> import glob 2024-09-17T03:15:06.613 DEBUG:teuthology.orchestra.run.smithi087:> import re 2024-09-17T03:15:06.613 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-09-17T03:15:06.613 DEBUG:teuthology.orchestra.run.smithi087:> import subprocess 2024-09-17T03:15:06.613 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:15:06.613 DEBUG:teuthology.orchestra.run.smithi087:> def _find_admin_socket(client_name): 2024-09-17T03:15:06.614 DEBUG:teuthology.orchestra.run.smithi087:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T03:15:06.614 DEBUG:teuthology.orchestra.run.smithi087:> files = glob.glob(asok_path) 2024-09-17T03:15:06.614 DEBUG:teuthology.orchestra.run.smithi087:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T03:15:06.614 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:15:06.614 DEBUG:teuthology.orchestra.run.smithi087:> # Given a non-glob path, it better be there 2024-09-17T03:15:06.614 DEBUG:teuthology.orchestra.run.smithi087:> if "*" not in asok_path: 2024-09-17T03:15:06.614 DEBUG:teuthology.orchestra.run.smithi087:> assert(len(files) == 1) 2024-09-17T03:15:06.614 DEBUG:teuthology.orchestra.run.smithi087:> return files[0] 2024-09-17T03:15:06.614 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:15:06.614 DEBUG:teuthology.orchestra.run.smithi087:> for f in files: 2024-09-17T03:15:06.614 DEBUG:teuthology.orchestra.run.smithi087:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T03:15:06.614 DEBUG:teuthology.orchestra.run.smithi087:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T03:15:06.614 DEBUG:teuthology.orchestra.run.smithi087:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T03:15:06.614 DEBUG:teuthology.orchestra.run.smithi087:> contents = proc_f.read() 2024-09-17T03:15:06.614 DEBUG:teuthology.orchestra.run.smithi087:> if mountpoint in contents: 2024-09-17T03:15:06.614 DEBUG:teuthology.orchestra.run.smithi087:> return f 2024-09-17T03:15:06.615 DEBUG:teuthology.orchestra.run.smithi087:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T03:15:06.615 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:15:06.615 DEBUG:teuthology.orchestra.run.smithi087:> print(_find_admin_socket("client.0")) 2024-09-17T03:15:06.615 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-09-17T03:15:06.729 INFO:teuthology.orchestra.run.smithi087.stdout:/var/run/ceph/ceph-client.0.49506.asok 2024-09-17T03:15:06.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:15:06.740 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49506.asok 2024-09-17T03:15:06.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:06.741 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.49506.asok status 2024-09-17T03:15:06.882 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-17T03:15:06.882 INFO:teuthology.orchestra.run.smithi087.stdout: "metadata": { 2024-09-17T03:15:06.882 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T03:15:06.883 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T03:15:06.883 INFO:teuthology.orchestra.run.smithi087.stdout: "entity_id": "0", 2024-09-17T03:15:06.883 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": "smithi087", 2024-09-17T03:15:06.883 INFO:teuthology.orchestra.run.smithi087.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T03:15:06.883 INFO:teuthology.orchestra.run.smithi087.stdout: "pid": "49506", 2024-09-17T03:15:06.883 INFO:teuthology.orchestra.run.smithi087.stdout: "root": "/" 2024-09-17T03:15:06.884 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:06.884 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_count": 0, 2024-09-17T03:15:06.884 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_pinned_count": 0, 2024-09-17T03:15:06.884 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 5220, 2024-09-17T03:15:06.884 INFO:teuthology.orchestra.run.smithi087.stdout: "inst": { 2024-09-17T03:15:06.885 INFO:teuthology.orchestra.run.smithi087.stdout: "name": { 2024-09-17T03:15:06.885 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "client", 2024-09-17T03:15:06.885 INFO:teuthology.orchestra.run.smithi087.stdout: "num": 5220 2024-09-17T03:15:06.885 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:06.885 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-09-17T03:15:06.885 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-09-17T03:15:06.886 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-09-17T03:15:06.886 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 912057571 2024-09-17T03:15:06.886 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:06.886 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:06.886 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-09-17T03:15:06.886 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-09-17T03:15:06.887 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-09-17T03:15:06.887 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 912057571 2024-09-17T03:15:06.887 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:06.887 INFO:teuthology.orchestra.run.smithi087.stdout: "inst_str": "client.5220 192.168.0.1:0/912057571", 2024-09-17T03:15:06.887 INFO:teuthology.orchestra.run.smithi087.stdout: "addr_str": "192.168.0.1:0/912057571", 2024-09-17T03:15:06.887 INFO:teuthology.orchestra.run.smithi087.stdout: "inode_count": 1, 2024-09-17T03:15:06.888 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_epoch": 13, 2024-09-17T03:15:06.888 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch": 34, 2024-09-17T03:15:06.888 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch_barrier": 0, 2024-09-17T03:15:06.888 INFO:teuthology.orchestra.run.smithi087.stdout: "blocklisted": false, 2024-09-17T03:15:06.888 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs" 2024-09-17T03:15:06.888 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-17T03:15:06.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:06.891 DEBUG:teuthology.orchestra.run.smithi087:> stat --file-system '--printf=%T 2024-09-17T03:15:06.891 DEBUG:teuthology.orchestra.run.smithi087:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:06.906 INFO:teuthology.orchestra.run.smithi087.stdout:fuseblk 2024-09-17T03:15:06.907 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:06.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:06.907 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:06.972 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create data_new --pg_num_min 64 2024-09-17T03:15:07.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.129+0000 7f9ab6030640 1 -- 172.21.15.87:0/2827365747 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ab00cfa70 msgr2=0x7f9ab00709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:07.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.129+0000 7f9ab6030640 1 --2- 172.21.15.87:0/2827365747 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ab00cfa70 0x7f9ab00709c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0005fd0 tx=0x7f9aa002ab40 comp rx=0 tx=0).stop 2024-09-17T03:15:07.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.129+0000 7f9ab6030640 1 -- 172.21.15.87:0/2827365747 shutdown_connections 2024-09-17T03:15:07.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.129+0000 7f9ab6030640 1 --2- 172.21.15.87:0/2827365747 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ab00cfa70 0x7f9ab00709c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:07.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.129+0000 7f9ab6030640 1 -- 172.21.15.87:0/2827365747 >> 172.21.15.87:0/2827365747 conn(0x7f9ab00d0060 msgr2=0x7f9ab00d0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:07.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.129+0000 7f9ab6030640 1 -- 172.21.15.87:0/2827365747 shutdown_connections 2024-09-17T03:15:07.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.129+0000 7f9ab6030640 1 -- 172.21.15.87:0/2827365747 wait complete. 2024-09-17T03:15:07.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.131+0000 7f9ab6030640 1 Processor -- start 2024-09-17T03:15:07.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.131+0000 7f9ab6030640 1 -- start start 2024-09-17T03:15:07.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.132+0000 7f9ab6030640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ab00ca6b0 0x7f9ab00b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:07.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.132+0000 7f9ab6030640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ab00cfa70 0x7f9ab00ae650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:07.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.132+0000 7f9ab6030640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ab015b750 0x7f9ab009cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:07.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.132+0000 7f9ab6030640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab015c600 con 0x7f9ab00ca6b0 2024-09-17T03:15:07.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.132+0000 7f9ab6030640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab015c480 con 0x7f9ab015b750 2024-09-17T03:15:07.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.132+0000 7f9ab6030640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9ab015c780 con 0x7f9ab00cfa70 2024-09-17T03:15:07.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.132+0000 7f9aaf7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ab00ca6b0 0x7f9ab00b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:07.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.132+0000 7f9aaf7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ab00ca6b0 0x7f9ab00b5aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34386/0 (socket says 172.21.15.87:34386) 2024-09-17T03:15:07.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.132+0000 7f9aaf7fe640 1 -- 172.21.15.87:0/4109161119 learned_addr learned my addr 172.21.15.87:0/4109161119 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:07.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.132+0000 7f9aaffff640 1 --2- 172.21.15.87:0/4109161119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ab015b750 0x7f9ab009cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:07.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.132+0000 7f9aaeffd640 1 --2- 172.21.15.87:0/4109161119 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ab00cfa70 0x7f9ab00ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:07.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.132+0000 7f9aaf7fe640 1 -- 172.21.15.87:0/4109161119 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ab00cfa70 msgr2=0x7f9ab00ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:07.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.132+0000 7f9aaf7fe640 1 --2- 172.21.15.87:0/4109161119 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ab00cfa70 0x7f9ab00ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:07.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.132+0000 7f9aaf7fe640 1 -- 172.21.15.87:0/4109161119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ab015b750 msgr2=0x7f9ab009cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:07.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.132+0000 7f9aaf7fe640 1 --2- 172.21.15.87:0/4109161119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ab015b750 0x7f9ab009cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:07.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.132+0000 7f9aaf7fe640 1 -- 172.21.15.87:0/4109161119 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab01f33e0 con 0x7f9ab00ca6b0 2024-09-17T03:15:07.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.132+0000 7f9aaffff640 1 --2- 172.21.15.87:0/4109161119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ab015b750 0x7f9ab009cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:07.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.133+0000 7f9aaf7fe640 1 --2- 172.21.15.87:0/4109161119 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ab00ca6b0 0x7f9ab00b5aa0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0009bf0 tx=0x7f9aa002c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:07.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.133+0000 7f9a97fff640 1 -- 172.21.15.87:0/4109161119 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0044070 con 0x7f9ab00ca6b0 2024-09-17T03:15:07.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.133+0000 7f9a97fff640 1 -- 172.21.15.87:0/4109161119 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9aa0004250 con 0x7f9ab00ca6b0 2024-09-17T03:15:07.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.133+0000 7f9ab6030640 1 -- 172.21.15.87:0/4109161119 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab01f3680 con 0x7f9ab00ca6b0 2024-09-17T03:15:07.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.133+0000 7f9a97fff640 1 -- 172.21.15.87:0/4109161119 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0004680 con 0x7f9ab00ca6b0 2024-09-17T03:15:07.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.133+0000 7f9ab6030640 1 -- 172.21.15.87:0/4109161119 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab01f3c40 con 0x7f9ab00ca6b0 2024-09-17T03:15:07.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.134+0000 7f9ab6030640 1 -- 172.21.15.87:0/4109161119 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ab00709c0 con 0x7f9ab00ca6b0 2024-09-17T03:15:07.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.134+0000 7f9a97fff640 1 -- 172.21.15.87:0/4109161119 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9aa0036400 con 0x7f9ab00ca6b0 2024-09-17T03:15:07.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.135+0000 7f9a97fff640 1 --2- 172.21.15.87:0/4109161119 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9a9c078bc0 0x7f9a9c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:07.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.135+0000 7f9a97fff640 1 -- 172.21.15.87:0/4109161119 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f9aa00bb6b0 con 0x7f9ab00ca6b0 2024-09-17T03:15:07.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.135+0000 7f9aaeffd640 1 --2- 172.21.15.87:0/4109161119 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9a9c078bc0 0x7f9a9c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:07.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.136+0000 7f9aaeffd640 1 --2- 172.21.15.87:0/4109161119 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9a9c078bc0 0x7f9a9c07b0d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4007fd0 tx=0x7f9aa4009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:07.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.139+0000 7f9a97fff640 1 -- 172.21.15.87:0/4109161119 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9aa0032030 con 0x7f9ab00ca6b0 2024-09-17T03:15:07.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.315+0000 7f9ab6030640 1 -- 172.21.15.87:0/4109161119 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64} v 0) -- 0x7f9ab015b090 con 0x7f9ab00ca6b0 2024-09-17T03:15:07.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.563+0000 7f9a97fff640 1 -- 172.21.15.87:0/4109161119 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "data_new", "pg_num_min": 64}]=0 pool 'data_new' created v35) ==== 124+0+0 (secure 0 0 0) 0x7f9aa0004820 con 0x7f9ab00ca6b0 2024-09-17T03:15:07.562 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'data_new' created 2024-09-17T03:15:07.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.565+0000 7f9ab6030640 1 -- 172.21.15.87:0/4109161119 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9a9c078bc0 msgr2=0x7f9a9c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:07.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.565+0000 7f9ab6030640 1 --2- 172.21.15.87:0/4109161119 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9a9c078bc0 0x7f9a9c07b0d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4007fd0 tx=0x7f9aa4009040 comp rx=0 tx=0).stop 2024-09-17T03:15:07.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.565+0000 7f9ab6030640 1 -- 172.21.15.87:0/4109161119 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ab00ca6b0 msgr2=0x7f9ab00b5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:07.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.565+0000 7f9ab6030640 1 --2- 172.21.15.87:0/4109161119 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ab00ca6b0 0x7f9ab00b5aa0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0009bf0 tx=0x7f9aa002c8b0 comp rx=0 tx=0).stop 2024-09-17T03:15:07.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.565+0000 7f9ab6030640 1 -- 172.21.15.87:0/4109161119 shutdown_connections 2024-09-17T03:15:07.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.565+0000 7f9ab6030640 1 --2- 172.21.15.87:0/4109161119 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9a9c078bc0 0x7f9a9c07b0d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:07.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.565+0000 7f9ab6030640 1 --2- 172.21.15.87:0/4109161119 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ab015b750 0x7f9ab009cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:07.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.565+0000 7f9ab6030640 1 --2- 172.21.15.87:0/4109161119 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ab00cfa70 0x7f9ab00ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:07.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.565+0000 7f9ab6030640 1 --2- 172.21.15.87:0/4109161119 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ab00ca6b0 0x7f9ab00b5aa0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:07.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.565+0000 7f9ab6030640 1 -- 172.21.15.87:0/4109161119 >> 172.21.15.87:0/4109161119 conn(0x7f9ab00d0060 msgr2=0x7f9ab015bb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:07.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.565+0000 7f9ab6030640 1 -- 172.21.15.87:0/4109161119 shutdown_connections 2024-09-17T03:15:07.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.566+0000 7f9ab6030640 1 -- 172.21.15.87:0/4109161119 wait complete. 2024-09-17T03:15:07.579 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs data_new 2024-09-17T03:15:07.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.695+0000 7f6ac440a640 1 -- 172.21.15.87:0/696074941 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6abc156cb0 msgr2=0x7f6abc0af150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:07.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.695+0000 7f6ac440a640 1 --2- 172.21.15.87:0/696074941 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6abc156cb0 0x7f6abc0af150 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8005fd0 tx=0x7f6ab802a5d0 comp rx=0 tx=0).stop 2024-09-17T03:15:07.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.695+0000 7f6ac440a640 1 -- 172.21.15.87:0/696074941 shutdown_connections 2024-09-17T03:15:07.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.695+0000 7f6ac440a640 1 --2- 172.21.15.87:0/696074941 >> v2:172.21.15.149:3300/0 conn(0x7f6abc160180 0x7f6abc160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:07.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.695+0000 7f6ac440a640 1 --2- 172.21.15.87:0/696074941 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6abc156cb0 0x7f6abc0af150 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:07.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.695+0000 7f6ac440a640 1 -- 172.21.15.87:0/696074941 >> 172.21.15.87:0/696074941 conn(0x7f6abc0b1c40 msgr2=0x7f6abc0b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:07.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.695+0000 7f6ac440a640 1 -- 172.21.15.87:0/696074941 shutdown_connections 2024-09-17T03:15:07.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.695+0000 7f6ac440a640 1 -- 172.21.15.87:0/696074941 wait complete. 2024-09-17T03:15:07.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.697+0000 7f6ac440a640 1 Processor -- start 2024-09-17T03:15:07.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.697+0000 7f6ac440a640 1 -- start start 2024-09-17T03:15:07.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.698+0000 7f6ac440a640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6abc156cb0 0x7f6abc1f7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:07.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.698+0000 7f6ac440a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6abc15cb90 0x7f6abc0d0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:07.698 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.698+0000 7f6ac440a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6abc160180 0x7f6abc1458c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:07.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.698+0000 7f6ac440a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc0af810 con 0x7f6abc160180 2024-09-17T03:15:07.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.698+0000 7f6ac440a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6abc0af690 con 0x7f6abc15cb90 2024-09-17T03:15:07.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.698+0000 7f6ac440a640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6abc15d6b0 con 0x7f6abc156cb0 2024-09-17T03:15:07.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.698+0000 7f6ac29a7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6abc160180 0x7f6abc1458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:07.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.699+0000 7f6ac29a7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6abc160180 0x7f6abc1458c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34394/0 (socket says 172.21.15.87:34394) 2024-09-17T03:15:07.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.699+0000 7f6ac29a7640 1 -- 172.21.15.87:0/2718168516 learned_addr learned my addr 172.21.15.87:0/2718168516 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:07.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.699+0000 7f6ac19a5640 1 --2- 172.21.15.87:0/2718168516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6abc15cb90 0x7f6abc0d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:07.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.699+0000 7f6ac21a6640 1 --2- 172.21.15.87:0/2718168516 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6abc156cb0 0x7f6abc1f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:07.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.699+0000 7f6ac29a7640 1 -- 172.21.15.87:0/2718168516 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6abc156cb0 msgr2=0x7f6abc1f7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:07.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.699+0000 7f6ac29a7640 1 --2- 172.21.15.87:0/2718168516 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6abc156cb0 0x7f6abc1f7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:07.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.699+0000 7f6ac29a7640 1 -- 172.21.15.87:0/2718168516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6abc15cb90 msgr2=0x7f6abc0d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:07.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.699+0000 7f6ac29a7640 1 --2- 172.21.15.87:0/2718168516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6abc15cb90 0x7f6abc0d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:07.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.699+0000 7f6ac29a7640 1 -- 172.21.15.87:0/2718168516 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6abc146510 con 0x7f6abc160180 2024-09-17T03:15:07.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.699+0000 7f6ac19a5640 1 --2- 172.21.15.87:0/2718168516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6abc15cb90 0x7f6abc0d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:07.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.699+0000 7f6ac29a7640 1 --2- 172.21.15.87:0/2718168516 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6abc160180 0x7f6abc1458c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6ab400c700 tx=0x7f6ab400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:07.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.699+0000 7f6ab2ffd640 1 -- 172.21.15.87:0/2718168516 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4003b70 con 0x7f6abc160180 2024-09-17T03:15:07.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.699+0000 7f6ac440a640 1 -- 172.21.15.87:0/2718168516 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6abc1467f0 con 0x7f6abc160180 2024-09-17T03:15:07.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.699+0000 7f6ab2ffd640 1 -- 172.21.15.87:0/2718168516 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ab4003d10 con 0x7f6abc160180 2024-09-17T03:15:07.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.699+0000 7f6ab2ffd640 1 -- 172.21.15.87:0/2718168516 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ab400aab0 con 0x7f6abc160180 2024-09-17T03:15:07.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.699+0000 7f6ac440a640 1 -- 172.21.15.87:0/2718168516 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6abc1f6d40 con 0x7f6abc160180 2024-09-17T03:15:07.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.700+0000 7f6ac440a640 1 -- 172.21.15.87:0/2718168516 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6abc146280 con 0x7f6abc160180 2024-09-17T03:15:07.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.701+0000 7f6ab2ffd640 1 -- 172.21.15.87:0/2718168516 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6ab400ac50 con 0x7f6abc160180 2024-09-17T03:15:07.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.702+0000 7f6ab2ffd640 1 --2- 172.21.15.87:0/2718168516 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6aa0078bc0 0x7f6aa007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:07.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.702+0000 7f6ab2ffd640 1 -- 172.21.15.87:0/2718168516 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 6072+0+0 (secure 0 0 0) 0x7f6ab409b6c0 con 0x7f6abc160180 2024-09-17T03:15:07.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.702+0000 7f6ac21a6640 1 --2- 172.21.15.87:0/2718168516 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6aa0078bc0 0x7f6aa007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:07.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.702+0000 7f6ac21a6640 1 --2- 172.21.15.87:0/2718168516 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6aa0078bc0 0x7f6aa007b0d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8033000 tx=0x7f6ab8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:07.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.706+0000 7f6ab2ffd640 1 -- 172.21.15.87:0/2718168516 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6ab4063990 con 0x7f6abc160180 2024-09-17T03:15:07.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:07.916+0000 7f6ac440a640 1 -- 172.21.15.87:0/2718168516 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"} v 0) -- 0x7f6abc1561d0 con 0x7f6abc160180 2024-09-17T03:15:08.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.574+0000 7f6ab2ffd640 1 -- 172.21.15.87:0/2718168516 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "data_new"}]=0 added data pool 6 to fsmap v14) ==== 131+0+0 (secure 0 0 0) 0x7f6ab4063290 con 0x7f6abc160180 2024-09-17T03:15:08.574 INFO:teuthology.orchestra.run.smithi087.stderr:added data pool 6 to fsmap 2024-09-17T03:15:08.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.580+0000 7f6ac440a640 1 -- 172.21.15.87:0/2718168516 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6aa0078bc0 msgr2=0x7f6aa007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:08.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.580+0000 7f6ac440a640 1 --2- 172.21.15.87:0/2718168516 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6aa0078bc0 0x7f6aa007b0d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8033000 tx=0x7f6ab8004730 comp rx=0 tx=0).stop 2024-09-17T03:15:08.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.580+0000 7f6ac440a640 1 -- 172.21.15.87:0/2718168516 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6abc160180 msgr2=0x7f6abc1458c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:08.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.580+0000 7f6ac440a640 1 --2- 172.21.15.87:0/2718168516 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6abc160180 0x7f6abc1458c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6ab400c700 tx=0x7f6ab400cbc0 comp rx=0 tx=0).stop 2024-09-17T03:15:08.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.580+0000 7f6ac440a640 1 -- 172.21.15.87:0/2718168516 shutdown_connections 2024-09-17T03:15:08.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.580+0000 7f6ac440a640 1 --2- 172.21.15.87:0/2718168516 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6aa0078bc0 0x7f6aa007b0d0 secure :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8033000 tx=0x7f6ab8004730 comp rx=0 tx=0).stop 2024-09-17T03:15:08.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.580+0000 7f6ac440a640 1 --2- 172.21.15.87:0/2718168516 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6abc160180 0x7f6abc1458c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:08.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.580+0000 7f6ac440a640 1 --2- 172.21.15.87:0/2718168516 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6abc15cb90 0x7f6abc0d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:08.580 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.580+0000 7f6ac440a640 1 --2- 172.21.15.87:0/2718168516 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6abc156cb0 0x7f6abc1f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:08.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.580+0000 7f6ac440a640 1 -- 172.21.15.87:0/2718168516 >> 172.21.15.87:0/2718168516 conn(0x7f6abc0b1c40 msgr2=0x7f6abc075ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:08.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.584+0000 7f6ac440a640 1 -- 172.21.15.87:0/2718168516 shutdown_connections 2024-09-17T03:15:08.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.584+0000 7f6ac440a640 1 -- 172.21.15.87:0/2718168516 wait complete. 2024-09-17T03:15:08.598 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:15:08.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.717+0000 7ffa735e6640 1 -- 172.21.15.87:0/437742655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa6c15b830 msgr2=0x7ffa6c0cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:08.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.717+0000 7ffa735e6640 1 --2- 172.21.15.87:0/437742655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa6c15b830 0x7ffa6c0cc7c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c005fd0 tx=0x7ffa5c02a530 comp rx=0 tx=0).stop 2024-09-17T03:15:08.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.717+0000 7ffa735e6640 1 -- 172.21.15.87:0/437742655 shutdown_connections 2024-09-17T03:15:08.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.717+0000 7ffa735e6640 1 --2- 172.21.15.87:0/437742655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa6c15b830 0x7ffa6c0cc7c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:08.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.717+0000 7ffa735e6640 1 --2- 172.21.15.87:0/437742655 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa6c15aed0 0x7ffa6c0cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:08.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.717+0000 7ffa735e6640 1 -- 172.21.15.87:0/437742655 >> 172.21.15.87:0/437742655 conn(0x7ffa6c06ee70 msgr2=0x7ffa6c06f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:08.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.718+0000 7ffa735e6640 1 -- 172.21.15.87:0/437742655 shutdown_connections 2024-09-17T03:15:08.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.718+0000 7ffa735e6640 1 -- 172.21.15.87:0/437742655 wait complete. 2024-09-17T03:15:08.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.719+0000 7ffa735e6640 1 Processor -- start 2024-09-17T03:15:08.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.719+0000 7ffa735e6640 1 -- start start 2024-09-17T03:15:08.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.721+0000 7ffa735e6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa6c0cffd0 0x7ffa6c1f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:08.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.721+0000 7ffa735e6640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa6c15aed0 0x7ffa6c151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:08.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.721+0000 7ffa735e6640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa6c15b830 0x7ffa6c0b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:08.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.721+0000 7ffa735e6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c070f80 con 0x7ffa6c0cffd0 2024-09-17T03:15:08.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.721+0000 7ffa735e6640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6c070e00 con 0x7ffa6c15b830 2024-09-17T03:15:08.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.721+0000 7ffa735e6640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ffa6c0c5a60 con 0x7ffa6c15aed0 2024-09-17T03:15:08.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.721+0000 7ffa71382640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa6c0cffd0 0x7ffa6c1f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:08.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.721+0000 7ffa71382640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa6c0cffd0 0x7ffa6c1f5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34410/0 (socket says 172.21.15.87:34410) 2024-09-17T03:15:08.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.721+0000 7ffa71382640 1 -- 172.21.15.87:0/2591826262 learned_addr learned my addr 172.21.15.87:0/2591826262 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:08.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.721+0000 7ffa71b83640 1 --2- 172.21.15.87:0/2591826262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa6c15b830 0x7ffa6c0b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:08.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.721+0000 7ffa70b81640 1 --2- 172.21.15.87:0/2591826262 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa6c15aed0 0x7ffa6c151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:08.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.721+0000 7ffa71382640 1 -- 172.21.15.87:0/2591826262 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa6c15aed0 msgr2=0x7ffa6c151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:08.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.721+0000 7ffa71382640 1 --2- 172.21.15.87:0/2591826262 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa6c15aed0 0x7ffa6c151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:08.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.721+0000 7ffa71382640 1 -- 172.21.15.87:0/2591826262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa6c15b830 msgr2=0x7ffa6c0b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:08.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.721+0000 7ffa71382640 1 --2- 172.21.15.87:0/2591826262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa6c15b830 0x7ffa6c0b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:08.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.721+0000 7ffa71382640 1 -- 172.21.15.87:0/2591826262 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa6c155590 con 0x7ffa6c0cffd0 2024-09-17T03:15:08.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.721+0000 7ffa71382640 1 --2- 172.21.15.87:0/2591826262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa6c0cffd0 0x7ffa6c1f5780 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800c970 tx=0x7ffa6800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:08.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.721+0000 7ffa61ffb640 1 -- 172.21.15.87:0/2591826262 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa68007bb0 con 0x7ffa6c0cffd0 2024-09-17T03:15:08.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.721+0000 7ffa61ffb640 1 -- 172.21.15.87:0/2591826262 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa68007e40 con 0x7ffa6c0cffd0 2024-09-17T03:15:08.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.721+0000 7ffa735e6640 1 -- 172.21.15.87:0/2591826262 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa6c155a30 con 0x7ffa6c0cffd0 2024-09-17T03:15:08.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.721+0000 7ffa61ffb640 1 -- 172.21.15.87:0/2591826262 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa68005280 con 0x7ffa6c0cffd0 2024-09-17T03:15:08.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.722+0000 7ffa735e6640 1 -- 172.21.15.87:0/2591826262 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa6c156440 con 0x7ffa6c0cffd0 2024-09-17T03:15:08.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.723+0000 7ffa735e6640 1 -- 172.21.15.87:0/2591826262 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa6c0ce590 con 0x7ffa6c0cffd0 2024-09-17T03:15:08.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.723+0000 7ffa61ffb640 1 -- 172.21.15.87:0/2591826262 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffa68005420 con 0x7ffa6c0cffd0 2024-09-17T03:15:08.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.724+0000 7ffa61ffb640 1 --2- 172.21.15.87:0/2591826262 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ffa50078bc0 0x7ffa5007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:08.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.724+0000 7ffa61ffb640 1 -- 172.21.15.87:0/2591826262 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6104+0+0 (secure 0 0 0) 0x7ffa6809b2d0 con 0x7ffa6c0cffd0 2024-09-17T03:15:08.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.724+0000 7ffa70b81640 1 --2- 172.21.15.87:0/2591826262 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ffa50078bc0 0x7ffa5007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:08.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.724+0000 7ffa70b81640 1 --2- 172.21.15.87:0/2591826262 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ffa50078bc0 0x7ffa5007b0d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c033000 tx=0x7ffa5c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:08.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.728+0000 7ffa61ffb640 1 -- 172.21.15.87:0/2591826262 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ffa68014030 con 0x7ffa6c0cffd0 2024-09-17T03:15:08.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.936+0000 7ffa735e6640 1 -- 172.21.15.87:0/2591826262 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa6c0ce940 con 0x7ffa6c0cffd0 2024-09-17T03:15:08.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.937+0000 7ffa61ffb640 1 -- 172.21.15.87:0/2591826262 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5228 (secure 0 0 0) 0x7ffa680636d0 con 0x7ffa6c0cffd0 2024-09-17T03:15:08.936 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:08.936 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":14,"btime":"2024-09-17T03:15:08:564652+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6836/1081794135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1081794135},{"type":"v1","addr":"172.21.15.149:6836","nonce":1081794135}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2152665509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6835","nonce":2152665509},{"type":"v1","addr":"172.21.15.149:6837","nonce":2152665509}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/704821751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":704821751},{"type":"v1","addr":"172.21.15.87:6835","nonce":704821751}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:14:55.675616+0000","modified":"2024-09-17T03:15:07.917703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5004},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5004":{"gid":5004,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6834/1243349323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":1243349323},{"type":"v1","addr":"172.21.15.87:6834","nonce":1243349323}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5004,"qdb_cluster":[5004]},"id":2}]} 2024-09-17T03:15:08.936 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 14 2024-09-17T03:15:08.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.939+0000 7ffa735e6640 1 -- 172.21.15.87:0/2591826262 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ffa50078bc0 msgr2=0x7ffa5007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:08.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.939+0000 7ffa735e6640 1 --2- 172.21.15.87:0/2591826262 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ffa50078bc0 0x7ffa5007b0d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c033000 tx=0x7ffa5c004730 comp rx=0 tx=0).stop 2024-09-17T03:15:08.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.939+0000 7ffa735e6640 1 -- 172.21.15.87:0/2591826262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa6c0cffd0 msgr2=0x7ffa6c1f5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:08.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.939+0000 7ffa735e6640 1 --2- 172.21.15.87:0/2591826262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa6c0cffd0 0x7ffa6c1f5780 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ffa6800c970 tx=0x7ffa6800ce30 comp rx=0 tx=0).stop 2024-09-17T03:15:08.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.939+0000 7ffa735e6640 1 -- 172.21.15.87:0/2591826262 shutdown_connections 2024-09-17T03:15:08.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.939+0000 7ffa735e6640 1 --2- 172.21.15.87:0/2591826262 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ffa50078bc0 0x7ffa5007b0d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:08.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.939+0000 7ffa735e6640 1 --2- 172.21.15.87:0/2591826262 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa6c15b830 0x7ffa6c0b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:08.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.939+0000 7ffa735e6640 1 --2- 172.21.15.87:0/2591826262 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa6c15aed0 0x7ffa6c151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:08.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.939+0000 7ffa735e6640 1 --2- 172.21.15.87:0/2591826262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa6c0cffd0 0x7ffa6c1f5780 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:08.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.939+0000 7ffa735e6640 1 -- 172.21.15.87:0/2591826262 >> 172.21.15.87:0/2591826262 conn(0x7ffa6c06ee70 msgr2=0x7ffa6c075c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:08.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.939+0000 7ffa735e6640 1 -- 172.21.15.87:0/2591826262 shutdown_connections 2024-09-17T03:15:08.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:08.939+0000 7ffa735e6640 1 -- 172.21.15.87:0/2591826262 wait complete. 2024-09-17T03:15:08.949 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:15:08.985 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:15:08.995 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:15:09.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.065+0000 7fc692d8b640 1 -- 172.21.15.87:0/4071551634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc68c0b0710 msgr2=0x7fc68c15f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:09.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.065+0000 7fc692d8b640 1 --2- 172.21.15.87:0/4071551634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc68c0b0710 0x7fc68c15f180 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009830 tx=0x7fc67c02a510 comp rx=0 tx=0).stop 2024-09-17T03:15:09.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.065+0000 7fc692d8b640 1 -- 172.21.15.87:0/4071551634 shutdown_connections 2024-09-17T03:15:09.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.065+0000 7fc692d8b640 1 --2- 172.21.15.87:0/4071551634 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc68c15f6c0 0x7fc68c15bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:09.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.065+0000 7fc692d8b640 1 --2- 172.21.15.87:0/4071551634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc68c0b0710 0x7fc68c15f180 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:09.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.065+0000 7fc692d8b640 1 -- 172.21.15.87:0/4071551634 >> 172.21.15.87:0/4071551634 conn(0x7fc68c0b2d10 msgr2=0x7fc68c0b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:09.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.065+0000 7fc692d8b640 1 -- 172.21.15.87:0/4071551634 shutdown_connections 2024-09-17T03:15:09.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.065+0000 7fc692d8b640 1 -- 172.21.15.87:0/4071551634 wait complete. 2024-09-17T03:15:09.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.066+0000 7fc692d8b640 1 Processor -- start 2024-09-17T03:15:09.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.066+0000 7fc692d8b640 1 -- start start 2024-09-17T03:15:09.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.068+0000 7fc692d8b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc68c071ed0 0x7fc68c155ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:09.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.068+0000 7fc692d8b640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc68c0b0710 0x7fc68c154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:09.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.068+0000 7fc692d8b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc68c15f6c0 0x7fc68c13c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:09.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.068+0000 7fc692d8b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c15c250 con 0x7fc68c15f6c0 2024-09-17T03:15:09.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.068+0000 7fc692d8b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c15c0d0 con 0x7fc68c071ed0 2024-09-17T03:15:09.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.068+0000 7fc692d8b640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc68c15c3d0 con 0x7fc68c0b0710 2024-09-17T03:15:09.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.068+0000 7fc691328640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc68c15f6c0 0x7fc68c13c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:09.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.068+0000 7fc691328640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc68c15f6c0 0x7fc68c13c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34422/0 (socket says 172.21.15.87:34422) 2024-09-17T03:15:09.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.068+0000 7fc691328640 1 -- 172.21.15.87:0/327703536 learned_addr learned my addr 172.21.15.87:0/327703536 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:09.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.068+0000 7fc690b27640 1 --2- 172.21.15.87:0/327703536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc68c071ed0 0x7fc68c155ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:09.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.068+0000 7fc683fff640 1 --2- 172.21.15.87:0/327703536 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc68c0b0710 0x7fc68c154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:09.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.068+0000 7fc691328640 1 -- 172.21.15.87:0/327703536 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc68c0b0710 msgr2=0x7fc68c154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:09.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.068+0000 7fc691328640 1 --2- 172.21.15.87:0/327703536 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc68c0b0710 0x7fc68c154490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:09.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.068+0000 7fc691328640 1 -- 172.21.15.87:0/327703536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc68c071ed0 msgr2=0x7fc68c155ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:09.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.068+0000 7fc691328640 1 --2- 172.21.15.87:0/327703536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc68c071ed0 0x7fc68c155ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:09.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.068+0000 7fc691328640 1 -- 172.21.15.87:0/327703536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68c1f2da0 con 0x7fc68c15f6c0 2024-09-17T03:15:09.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.068+0000 7fc691328640 1 --2- 172.21.15.87:0/327703536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc68c15f6c0 0x7fc68c13c320 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc68400efc0 tx=0x7fc68400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:09.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.069+0000 7fc6817fa640 1 -- 172.21.15.87:0/327703536 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684003970 con 0x7fc68c15f6c0 2024-09-17T03:15:09.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.069+0000 7fc692d8b640 1 -- 172.21.15.87:0/327703536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc68c1f3080 con 0x7fc68c15f6c0 2024-09-17T03:15:09.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.069+0000 7fc6817fa640 1 -- 172.21.15.87:0/327703536 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc684003c00 con 0x7fc68c15f6c0 2024-09-17T03:15:09.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.069+0000 7fc6817fa640 1 -- 172.21.15.87:0/327703536 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc684004020 con 0x7fc68c15f6c0 2024-09-17T03:15:09.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.069+0000 7fc692d8b640 1 -- 172.21.15.87:0/327703536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc68c1f3640 con 0x7fc68c15f6c0 2024-09-17T03:15:09.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.070+0000 7fc692d8b640 1 -- 172.21.15.87:0/327703536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc68c0b2ae0 con 0x7fc68c15f6c0 2024-09-17T03:15:09.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.071+0000 7fc6817fa640 1 -- 172.21.15.87:0/327703536 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc6840041c0 con 0x7fc68c15f6c0 2024-09-17T03:15:09.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.071+0000 7fc6817fa640 1 --2- 172.21.15.87:0/327703536 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc678078b90 0x7fc67807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:09.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.071+0000 7fc6817fa640 1 -- 172.21.15.87:0/327703536 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6104+0+0 (secure 0 0 0) 0x7fc68409b8a0 con 0x7fc68c15f6c0 2024-09-17T03:15:09.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.071+0000 7fc690b27640 1 --2- 172.21.15.87:0/327703536 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc678078b90 0x7fc67807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:09.078 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.072+0000 7fc690b27640 1 --2- 172.21.15.87:0/327703536 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc678078b90 0x7fc67807b0a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc6880015e0 tx=0x7fc688009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:09.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.076+0000 7fc6817fa640 1 -- 172.21.15.87:0/327703536 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc68c0b2ae0 con 0x7fc68c15f6c0 2024-09-17T03:15:09.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.251+0000 7fc692d8b640 1 -- 172.21.15.87:0/327703536 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc68c156090 con 0x7fc68c15f6c0 2024-09-17T03:15:09.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.252+0000 7fc6817fa640 1 -- 172.21.15.87:0/327703536 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) ==== 74+0+18538 (secure 0 0 0) 0x7fc684063b60 con 0x7fc68c15f6c0 2024-09-17T03:15:09.252 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:09.252 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":36,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:15:07.917459+0000","last_up_change":"2024-09-17T03:14:01.833605+0000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:14:03.433404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:14:53.459693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T03:14:54.645011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-09-17T03:15:07.319575+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6825","nonce":2430409749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6827","nonce":2430409749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6831","nonce":2430409749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6829","nonce":2430409749}]},"public_addr":"172.21.15.87:6825/2430409749","cluster_addr":"172.21.15.87:6827/2430409749","heartbeat_back_addr":"172.21.15.87:6831/2430409749","heartbeat_front_addr":"172.21.15.87:6829/2430409749","state":["exists","up"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6809","nonce":3467605435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6811","nonce":3467605435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6815","nonce":3467605435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6813","nonce":3467605435}]},"public_addr":"172.21.15.87:6809/3467605435","cluster_addr":"172.21.15.87:6811/3467605435","heartbeat_back_addr":"172.21.15.87:6815/3467605435","heartbeat_front_addr":"172.21.15.87:6813/3467605435","state":["exists","up"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6801","nonce":1450834766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6803","nonce":1450834766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6807","nonce":1450834766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6805","nonce":1450834766}]},"public_addr":"172.21.15.87:6801/1450834766","cluster_addr":"172.21.15.87:6803/1450834766","heartbeat_back_addr":"172.21.15.87:6807/1450834766","heartbeat_front_addr":"172.21.15.87:6805/1450834766","state":["exists","up"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6817","nonce":4253161216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6819","nonce":4253161216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6823","nonce":4253161216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6821","nonce":4253161216}]},"public_addr":"172.21.15.87:6817/4253161216","cluster_addr":"172.21.15.87:6819/4253161216","heartbeat_back_addr":"172.21.15.87:6823/4253161216","heartbeat_front_addr":"172.21.15.87:6821/4253161216","state":["exists","up"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6803","nonce":2414907089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6805","nonce":2414907089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6809","nonce":2414907089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6807","nonce":2414907089}]},"public_addr":"172.21.15.149:6803/2414907089","cluster_addr":"172.21.15.149:6805/2414907089","heartbeat_back_addr":"172.21.15.149:6809/2414907089","heartbeat_front_addr":"172.21.15.149:6807/2414907089","state":["exists","up"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6827","nonce":2182999530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6829","nonce":2182999530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6833","nonce":2182999530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6831","nonce":2182999530}]},"public_addr":"172.21.15.149:6827/2182999530","cluster_addr":"172.21.15.149:6829/2182999530","heartbeat_back_addr":"172.21.15.149:6833/2182999530","heartbeat_front_addr":"172.21.15.149:6831/2182999530","state":["exists","up"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6811","nonce":4289519146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6813","nonce":4289519146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6817","nonce":4289519146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6815","nonce":4289519146}]},"public_addr":"172.21.15.149:6811/4289519146","cluster_addr":"172.21.15.149:6813/4289519146","heartbeat_back_addr":"172.21.15.149:6817/4289519146","heartbeat_front_addr":"172.21.15.149:6815/4289519146","state":["exists","up"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6819","nonce":2771010477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6821","nonce":2771010477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6825","nonce":2771010477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6823","nonce":2771010477}]},"public_addr":"172.21.15.149:6819/2771010477","cluster_addr":"172.21.15.149:6821/2771010477","heartbeat_back_addr":"172.21.15.149:6825/2771010477","heartbeat_front_addr":"172.21.15.149:6823/2771010477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.313297+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.642861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.654781+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.302606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.385791+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.915266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.121365+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.394432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:15:09.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.254+0000 7fc692d8b640 1 -- 172.21.15.87:0/327703536 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc678078b90 msgr2=0x7fc67807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:09.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.254+0000 7fc692d8b640 1 --2- 172.21.15.87:0/327703536 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc678078b90 0x7fc67807b0a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc6880015e0 tx=0x7fc688009040 comp rx=0 tx=0).stop 2024-09-17T03:15:09.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.254+0000 7fc692d8b640 1 -- 172.21.15.87:0/327703536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc68c15f6c0 msgr2=0x7fc68c13c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:09.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.254+0000 7fc692d8b640 1 --2- 172.21.15.87:0/327703536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc68c15f6c0 0x7fc68c13c320 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc68400efc0 tx=0x7fc68400c800 comp rx=0 tx=0).stop 2024-09-17T03:15:09.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.255+0000 7fc692d8b640 1 -- 172.21.15.87:0/327703536 shutdown_connections 2024-09-17T03:15:09.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.255+0000 7fc692d8b640 1 --2- 172.21.15.87:0/327703536 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc678078b90 0x7fc67807b0a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:09.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.255+0000 7fc692d8b640 1 --2- 172.21.15.87:0/327703536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc68c15f6c0 0x7fc68c13c320 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:09.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.255+0000 7fc692d8b640 1 --2- 172.21.15.87:0/327703536 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc68c0b0710 0x7fc68c154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:09.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.255+0000 7fc692d8b640 1 --2- 172.21.15.87:0/327703536 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc68c071ed0 0x7fc68c155ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:09.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.255+0000 7fc692d8b640 1 -- 172.21.15.87:0/327703536 >> 172.21.15.87:0/327703536 conn(0x7fc68c0b2d10 msgr2=0x7fc68c152000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:09.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.255+0000 7fc692d8b640 1 -- 172.21.15.87:0/327703536 shutdown_connections 2024-09-17T03:15:09.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.255+0000 7fc692d8b640 1 -- 172.21.15.87:0/327703536 wait complete. 2024-09-17T03:15:09.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:09.265 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch layoutfile) 2024-09-17T03:15:09.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:09.325 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir layoutdir) 2024-09-17T03:15:09.383 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rw' mon 'allow r' osd 'allow rw pool=cephfs_data,allow rw pool=data_new' 2024-09-17T03:15:09.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.544+0000 7fedff6f9640 1 -- 172.21.15.87:0/351714784 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedf80d1fe0 msgr2=0x7fedf80c5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:09.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.544+0000 7fedff6f9640 1 --2- 172.21.15.87:0/351714784 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedf80d1fe0 0x7fedf80c5de0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fedf4005fd0 tx=0x7fedf402a5d0 comp rx=0 tx=0).stop 2024-09-17T03:15:09.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.544+0000 7fedff6f9640 1 -- 172.21.15.87:0/351714784 shutdown_connections 2024-09-17T03:15:09.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.544+0000 7fedff6f9640 1 --2- 172.21.15.87:0/351714784 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fedf815c5d0 0x7fedf8160c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:09.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.544+0000 7fedff6f9640 1 --2- 172.21.15.87:0/351714784 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedf80d1fe0 0x7fedf80c5de0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:09.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.544+0000 7fedff6f9640 1 -- 172.21.15.87:0/351714784 >> 172.21.15.87:0/351714784 conn(0x7fedf8069120 msgr2=0x7fedf8069530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:09.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.544+0000 7fedff6f9640 1 -- 172.21.15.87:0/351714784 shutdown_connections 2024-09-17T03:15:09.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.544+0000 7fedff6f9640 1 -- 172.21.15.87:0/351714784 wait complete. 2024-09-17T03:15:09.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.546+0000 7fedff6f9640 1 Processor -- start 2024-09-17T03:15:09.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.546+0000 7fedff6f9640 1 -- start start 2024-09-17T03:15:09.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.547+0000 7fedff6f9640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fedf80ccaf0 0x7fedf806f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:09.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.547+0000 7fedff6f9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedf80d1fe0 0x7fedf81f9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:09.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.547+0000 7fedff6f9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedf815c5d0 0x7fedf8157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:09.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.547+0000 7fedff6f9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fedf80d0a90 con 0x7fedf80d1fe0 2024-09-17T03:15:09.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.547+0000 7fedff6f9640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fedf80d0910 con 0x7fedf815c5d0 2024-09-17T03:15:09.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.547+0000 7fedff6f9640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fedf80d0c10 con 0x7fedf80ccaf0 2024-09-17T03:15:09.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.547+0000 7fedfcc94640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedf80d1fe0 0x7fedf81f9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:09.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.547+0000 7fedfcc94640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedf80d1fe0 0x7fedf81f9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34452/0 (socket says 172.21.15.87:34452) 2024-09-17T03:15:09.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.547+0000 7fedfcc94640 1 -- 172.21.15.87:0/1909342824 learned_addr learned my addr 172.21.15.87:0/1909342824 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:09.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.547+0000 7fedfdc96640 1 --2- 172.21.15.87:0/1909342824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedf815c5d0 0x7fedf8157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:09.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.547+0000 7fedfd495640 1 --2- 172.21.15.87:0/1909342824 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fedf80ccaf0 0x7fedf806f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:09.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.547+0000 7fedfcc94640 1 -- 172.21.15.87:0/1909342824 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fedf80ccaf0 msgr2=0x7fedf806f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:09.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.547+0000 7fedfcc94640 1 --2- 172.21.15.87:0/1909342824 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fedf80ccaf0 0x7fedf806f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:09.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.547+0000 7fedfcc94640 1 -- 172.21.15.87:0/1909342824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedf815c5d0 msgr2=0x7fedf8157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:09.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.547+0000 7fedfcc94640 1 --2- 172.21.15.87:0/1909342824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedf815c5d0 0x7fedf8157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:09.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.547+0000 7fedfcc94640 1 -- 172.21.15.87:0/1909342824 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedf81f9d40 con 0x7fedf80d1fe0 2024-09-17T03:15:09.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.548+0000 7fedfcc94640 1 --2- 172.21.15.87:0/1909342824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedf80d1fe0 0x7fedf81f9800 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fede800d6a0 tx=0x7fede800db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:09.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.548+0000 7fededffb640 1 -- 172.21.15.87:0/1909342824 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede8017070 con 0x7fedf80d1fe0 2024-09-17T03:15:09.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.548+0000 7fededffb640 1 -- 172.21.15.87:0/1909342824 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fede8004310 con 0x7fedf80d1fe0 2024-09-17T03:15:09.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.548+0000 7fededffb640 1 -- 172.21.15.87:0/1909342824 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede8005280 con 0x7fedf80d1fe0 2024-09-17T03:15:09.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.548+0000 7fedff6f9640 1 -- 172.21.15.87:0/1909342824 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedf81f9020 con 0x7fedf80d1fe0 2024-09-17T03:15:09.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.548+0000 7fedff6f9640 1 -- 172.21.15.87:0/1909342824 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedf806fd00 con 0x7fedf80d1fe0 2024-09-17T03:15:09.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.549+0000 7fedff6f9640 1 -- 172.21.15.87:0/1909342824 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedf806f790 con 0x7fedf80d1fe0 2024-09-17T03:15:09.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.549+0000 7fededffb640 1 -- 172.21.15.87:0/1909342824 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fede8005420 con 0x7fedf80d1fe0 2024-09-17T03:15:09.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.550+0000 7fededffb640 1 --2- 172.21.15.87:0/1909342824 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7feddc078b90 0x7feddc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:09.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.550+0000 7fededffb640 1 -- 172.21.15.87:0/1909342824 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 6104+0+0 (secure 0 0 0) 0x7fede809b170 con 0x7fedf80d1fe0 2024-09-17T03:15:09.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.550+0000 7fedfd495640 1 --2- 172.21.15.87:0/1909342824 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7feddc078b90 0x7feddc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:09.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.551+0000 7fedfd495640 1 --2- 172.21.15.87:0/1909342824 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7feddc078b90 0x7feddc07b0a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fedf4033000 tx=0x7fedf4004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:09.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.554+0000 7fededffb640 1 -- 172.21.15.87:0/1909342824 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fede8020020 con 0x7fedf80d1fe0 2024-09-17T03:15:09.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.801+0000 7fedff6f9640 1 -- 172.21.15.87:0/1909342824 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]} v 0) -- 0x7fedf80d10d0 con 0x7fedf80d1fe0 2024-09-17T03:15:09.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.805+0000 7fededffb640 1 -- 172.21.15.87:0/1909342824 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rw", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v14) ==== 210+0+167 (secure 0 0 0) 0x7fede8062e50 con 0x7fedf80d1fe0 2024-09-17T03:15:09.804 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-09-17T03:15:09.805 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDH8+hmLskNLhAAd2M5ynGP8W6Pim11F18wYA== 2024-09-17T03:15:09.805 INFO:teuthology.orchestra.run.smithi087.stdout: caps mds = "allow rw" 2024-09-17T03:15:09.805 INFO:teuthology.orchestra.run.smithi087.stdout: caps mon = "allow r" 2024-09-17T03:15:09.805 INFO:teuthology.orchestra.run.smithi087.stdout: caps osd = "allow rw pool=cephfs_data,allow rw pool=data_new" 2024-09-17T03:15:09.805 INFO:teuthology.orchestra.run.smithi087.stderr:updated caps for client.0 2024-09-17T03:15:09.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.808+0000 7fedff6f9640 1 -- 172.21.15.87:0/1909342824 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7feddc078b90 msgr2=0x7feddc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:09.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.808+0000 7fedff6f9640 1 --2- 172.21.15.87:0/1909342824 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7feddc078b90 0x7feddc07b0a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fedf4033000 tx=0x7fedf4004730 comp rx=0 tx=0).stop 2024-09-17T03:15:09.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.808+0000 7fedff6f9640 1 -- 172.21.15.87:0/1909342824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedf80d1fe0 msgr2=0x7fedf81f9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:09.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.808+0000 7fedff6f9640 1 --2- 172.21.15.87:0/1909342824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedf80d1fe0 0x7fedf81f9800 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fede800d6a0 tx=0x7fede800db60 comp rx=0 tx=0).stop 2024-09-17T03:15:09.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.808+0000 7fedff6f9640 1 -- 172.21.15.87:0/1909342824 shutdown_connections 2024-09-17T03:15:09.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.808+0000 7fedff6f9640 1 --2- 172.21.15.87:0/1909342824 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7feddc078b90 0x7feddc07b0a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:09.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.808+0000 7fedff6f9640 1 --2- 172.21.15.87:0/1909342824 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fedf815c5d0 0x7fedf8157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:09.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.808+0000 7fedff6f9640 1 --2- 172.21.15.87:0/1909342824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fedf80d1fe0 0x7fedf81f9800 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:09.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.808+0000 7fedff6f9640 1 --2- 172.21.15.87:0/1909342824 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fedf80ccaf0 0x7fedf806f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:09.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.808+0000 7fedff6f9640 1 -- 172.21.15.87:0/1909342824 >> 172.21.15.87:0/1909342824 conn(0x7fedf8069120 msgr2=0x7fedf80d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:09.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.808+0000 7fedff6f9640 1 -- 172.21.15.87:0/1909342824 shutdown_connections 2024-09-17T03:15:09.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.808+0000 7fedff6f9640 1 -- 172.21.15.87:0/1909342824 wait complete. 2024-09-17T03:15:09.818 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:15:09.818 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:15:09.834 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:15:09.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.989+0000 7fdb8fd39640 1 -- 172.21.15.87:0/1512488102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb8815c6c0 msgr2=0x7fdb88160c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:09.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.989+0000 7fdb8fd39640 1 --2- 172.21.15.87:0/1512488102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb8815c6c0 0x7fdb88160c20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fdb80009830 tx=0x7fdb8002a480 comp rx=0 tx=0).stop 2024-09-17T03:15:09.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.989+0000 7fdb8fd39640 1 -- 172.21.15.87:0/1512488102 shutdown_connections 2024-09-17T03:15:09.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.989+0000 7fdb8fd39640 1 --2- 172.21.15.87:0/1512488102 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdb880b3d00 0x7fdb880b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:09.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.989+0000 7fdb8fd39640 1 --2- 172.21.15.87:0/1512488102 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb8815c6c0 0x7fdb88160c20 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:09.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.989+0000 7fdb8fd39640 1 -- 172.21.15.87:0/1512488102 >> 172.21.15.87:0/1512488102 conn(0x7fdb88069360 msgr2=0x7fdb880ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:09.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.989+0000 7fdb8fd39640 1 -- 172.21.15.87:0/1512488102 shutdown_connections 2024-09-17T03:15:09.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.989+0000 7fdb8fd39640 1 -- 172.21.15.87:0/1512488102 wait complete. 2024-09-17T03:15:09.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.990+0000 7fdb8fd39640 1 Processor -- start 2024-09-17T03:15:09.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.990+0000 7fdb8fd39640 1 -- start start 2024-09-17T03:15:09.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.992+0000 7fdb8fd39640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb880710e0 0x7fdb881ea0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:09.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.992+0000 7fdb8fd39640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb880b3d00 0x7fdb88158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:09.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.992+0000 7fdb8fd39640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdb8815c6c0 0x7fdb8815ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:09.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.992+0000 7fdb8fd39640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdb880d1470 con 0x7fdb880b3d00 2024-09-17T03:15:09.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.992+0000 7fdb8fd39640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdb880b80d0 con 0x7fdb880710e0 2024-09-17T03:15:09.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.992+0000 7fdb8fd39640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fdb880d15f0 con 0x7fdb8815c6c0 2024-09-17T03:15:09.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.992+0000 7fdb8d2d4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb880b3d00 0x7fdb88158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:09.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.992+0000 7fdb8d2d4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb880b3d00 0x7fdb88158ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34468/0 (socket says 172.21.15.87:34468) 2024-09-17T03:15:09.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.992+0000 7fdb8d2d4640 1 -- 172.21.15.87:0/679046657 learned_addr learned my addr 172.21.15.87:0/679046657 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:09.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.992+0000 7fdb8e2d6640 1 --2- 172.21.15.87:0/679046657 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdb8815c6c0 0x7fdb8815ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:09.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.992+0000 7fdb8dad5640 1 --2- 172.21.15.87:0/679046657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb880710e0 0x7fdb881ea0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:09.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.992+0000 7fdb8d2d4640 1 -- 172.21.15.87:0/679046657 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdb8815c6c0 msgr2=0x7fdb8815ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:09.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.992+0000 7fdb8d2d4640 1 --2- 172.21.15.87:0/679046657 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdb8815c6c0 0x7fdb8815ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:09.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.992+0000 7fdb8d2d4640 1 -- 172.21.15.87:0/679046657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb880710e0 msgr2=0x7fdb881ea0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:09.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.992+0000 7fdb8d2d4640 1 --2- 172.21.15.87:0/679046657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb880710e0 0x7fdb881ea0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:09.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.992+0000 7fdb8d2d4640 1 -- 172.21.15.87:0/679046657 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb881593e0 con 0x7fdb880b3d00 2024-09-17T03:15:09.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.992+0000 7fdb8dad5640 1 --2- 172.21.15.87:0/679046657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb880710e0 0x7fdb881ea0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:09.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.992+0000 7fdb8d2d4640 1 --2- 172.21.15.87:0/679046657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb880b3d00 0x7fdb88158ea0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800d6d0 tx=0x7fdb7800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:09.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.992+0000 7fdb7e7fc640 1 -- 172.21.15.87:0/679046657 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb78017070 con 0x7fdb880b3d00 2024-09-17T03:15:09.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.992+0000 7fdb7e7fc640 1 -- 172.21.15.87:0/679046657 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb78009b10 con 0x7fdb880b3d00 2024-09-17T03:15:09.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.992+0000 7fdb8fd39640 1 -- 172.21.15.87:0/679046657 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb881e9300 con 0x7fdb880b3d00 2024-09-17T03:15:09.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.992+0000 7fdb7e7fc640 1 -- 172.21.15.87:0/679046657 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb78010460 con 0x7fdb880b3d00 2024-09-17T03:15:09.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.993+0000 7fdb8fd39640 1 -- 172.21.15.87:0/679046657 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb881e98c0 con 0x7fdb880b3d00 2024-09-17T03:15:09.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.993+0000 7fdb8fd39640 1 -- 172.21.15.87:0/679046657 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb881f76e0 con 0x7fdb880b3d00 2024-09-17T03:15:09.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.994+0000 7fdb7e7fc640 1 -- 172.21.15.87:0/679046657 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdb7801a070 con 0x7fdb880b3d00 2024-09-17T03:15:09.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.995+0000 7fdb7e7fc640 1 --2- 172.21.15.87:0/679046657 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdb6c078b90 0x7fdb6c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:09.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.995+0000 7fdb7e7fc640 1 -- 172.21.15.87:0/679046657 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7fdb7809af30 con 0x7fdb880b3d00 2024-09-17T03:15:09.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.995+0000 7fdb8dad5640 1 --2- 172.21.15.87:0/679046657 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdb6c078b90 0x7fdb6c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:09.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.995+0000 7fdb8dad5640 1 --2- 172.21.15.87:0/679046657 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdb6c078b90 0x7fdb6c07b0a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdb80034000 tx=0x7fdb80004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:09.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:09.999+0000 7fdb7e7fc640 1 -- 172.21.15.87:0/679046657 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fdb78062b90 con 0x7fdb880b3d00 2024-09-17T03:15:10.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.183+0000 7fdb8fd39640 1 -- 172.21.15.87:0/679046657 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdb88071ce0 con 0x7fdb880b3d00 2024-09-17T03:15:10.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.184+0000 7fdb7e7fc640 1 -- 172.21.15.87:0/679046657 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) ==== 80+0+0 (secure 0 0 0) 0x7fdb780675c0 con 0x7fdb880b3d00 2024-09-17T03:15:10.182 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-09-17T03:15:10.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.186+0000 7fdb8fd39640 1 -- 172.21.15.87:0/679046657 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdb6c078b90 msgr2=0x7fdb6c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:10.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.186+0000 7fdb8fd39640 1 --2- 172.21.15.87:0/679046657 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdb6c078b90 0x7fdb6c07b0a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fdb80034000 tx=0x7fdb80004730 comp rx=0 tx=0).stop 2024-09-17T03:15:10.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.186+0000 7fdb8fd39640 1 -- 172.21.15.87:0/679046657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb880b3d00 msgr2=0x7fdb88158ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:10.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.186+0000 7fdb8fd39640 1 --2- 172.21.15.87:0/679046657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb880b3d00 0x7fdb88158ea0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800d6d0 tx=0x7fdb7800db90 comp rx=0 tx=0).stop 2024-09-17T03:15:10.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.186+0000 7fdb8fd39640 1 -- 172.21.15.87:0/679046657 shutdown_connections 2024-09-17T03:15:10.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.186+0000 7fdb8fd39640 1 --2- 172.21.15.87:0/679046657 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdb6c078b90 0x7fdb6c07b0a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:10.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.186+0000 7fdb8fd39640 1 --2- 172.21.15.87:0/679046657 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdb8815c6c0 0x7fdb8815ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:10.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.186+0000 7fdb8fd39640 1 --2- 172.21.15.87:0/679046657 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb880b3d00 0x7fdb88158ea0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:10.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.186+0000 7fdb8fd39640 1 --2- 172.21.15.87:0/679046657 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdb880710e0 0x7fdb881ea0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:10.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.186+0000 7fdb8fd39640 1 -- 172.21.15.87:0/679046657 >> 172.21.15.87:0/679046657 conn(0x7fdb88069360 msgr2=0x7fdb880c5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:10.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.186+0000 7fdb8fd39640 1 -- 172.21.15.87:0/679046657 shutdown_connections 2024-09-17T03:15:10.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.186+0000 7fdb8fd39640 1 -- 172.21.15.87:0/679046657 wait complete. 2024-09-17T03:15:10.196 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:15:10.196 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:15:10.212 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:15:10.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.369+0000 7ff8c2ede640 1 -- 172.21.15.87:0/419973554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8a4004d40 msgr2=0x7ff8a4005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:10.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.369+0000 7ff8c2ede640 1 --2- 172.21.15.87:0/419973554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8a4004d40 0x7ff8a4005150 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc074cb0 tx=0x7ff8a4005830 comp rx=0 tx=0).stop 2024-09-17T03:15:10.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.369+0000 7ff8c147b640 1 -- 172.21.15.87:0/419973554 reap_dead start 2024-09-17T03:15:10.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.369+0000 7ff8c2ede640 1 -- 172.21.15.87:0/419973554 shutdown_connections 2024-09-17T03:15:10.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.369+0000 7ff8c2ede640 1 -- 172.21.15.87:0/419973554 >> 172.21.15.87:0/419973554 conn(0x7ff8bc0d2230 msgr2=0x7ff8bc0d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:10.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.369+0000 7ff8c2ede640 1 -- 172.21.15.87:0/419973554 shutdown_connections 2024-09-17T03:15:10.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.369+0000 7ff8c2ede640 1 -- 172.21.15.87:0/419973554 wait complete. 2024-09-17T03:15:10.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.371+0000 7ff8c2ede640 1 Processor -- start 2024-09-17T03:15:10.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.371+0000 7ff8c2ede640 1 -- start start 2024-09-17T03:15:10.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.372+0000 7ff8c2ede640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8bc1e0120 0x7ff8bc1df7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:10.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.372+0000 7ff8c2ede640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff8bc1f6c40 0x7ff8bc0b60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:10.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.372+0000 7ff8c2ede640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8bc0b65f0 0x7ff8bc1f3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:10.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.372+0000 7ff8c2ede640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc0d1250 con 0x7ff8bc1e0120 2024-09-17T03:15:10.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.372+0000 7ff8c2ede640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc0b30d0 con 0x7ff8bc0b65f0 2024-09-17T03:15:10.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.372+0000 7ff8c2ede640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff8bc0d13d0 con 0x7ff8bc1f6c40 2024-09-17T03:15:10.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.372+0000 7ff8c0c7a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8bc1e0120 0x7ff8bc1df7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:10.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.372+0000 7ff8c0c7a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8bc1e0120 0x7ff8bc1df7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34492/0 (socket says 172.21.15.87:34492) 2024-09-17T03:15:10.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.372+0000 7ff8c0c7a640 1 -- 172.21.15.87:0/3915030405 learned_addr learned my addr 172.21.15.87:0/3915030405 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:10.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.372+0000 7ff8b3fff640 1 --2- 172.21.15.87:0/3915030405 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff8bc1f6c40 0x7ff8bc0b60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:10.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.372+0000 7ff8c147b640 1 --2- 172.21.15.87:0/3915030405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8bc0b65f0 0x7ff8bc1f3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:10.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.373+0000 7ff8c0c7a640 1 -- 172.21.15.87:0/3915030405 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff8bc1f6c40 msgr2=0x7ff8bc0b60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:10.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.373+0000 7ff8c0c7a640 1 --2- 172.21.15.87:0/3915030405 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff8bc1f6c40 0x7ff8bc0b60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:10.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.373+0000 7ff8c0c7a640 1 -- 172.21.15.87:0/3915030405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8bc0b65f0 msgr2=0x7ff8bc1f3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:10.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.373+0000 7ff8c0c7a640 1 --2- 172.21.15.87:0/3915030405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8bc0b65f0 0x7ff8bc1f3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:10.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.373+0000 7ff8c0c7a640 1 -- 172.21.15.87:0/3915030405 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8bc0b6a00 con 0x7ff8bc1e0120 2024-09-17T03:15:10.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.373+0000 7ff8b3fff640 1 --2- 172.21.15.87:0/3915030405 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff8bc1f6c40 0x7ff8bc0b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:10.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.373+0000 7ff8c0c7a640 1 --2- 172.21.15.87:0/3915030405 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8bc1e0120 0x7ff8bc1df7d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8011ba0 tx=0x7ff8b800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:10.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.373+0000 7ff8b17fa640 1 -- 172.21.15.87:0/3915030405 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b801e070 con 0x7ff8bc1e0120 2024-09-17T03:15:10.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.373+0000 7ff8b17fa640 1 -- 172.21.15.87:0/3915030405 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8b800e7c0 con 0x7ff8bc1e0120 2024-09-17T03:15:10.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.373+0000 7ff8c2ede640 1 -- 172.21.15.87:0/3915030405 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8bc1f7200 con 0x7ff8bc1e0120 2024-09-17T03:15:10.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.373+0000 7ff8b17fa640 1 -- 172.21.15.87:0/3915030405 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b800ebd0 con 0x7ff8bc1e0120 2024-09-17T03:15:10.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.373+0000 7ff8c2ede640 1 -- 172.21.15.87:0/3915030405 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8bc1f77c0 con 0x7ff8bc1e0120 2024-09-17T03:15:10.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.374+0000 7ff8c2ede640 1 -- 172.21.15.87:0/3915030405 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8bc0ce790 con 0x7ff8bc1e0120 2024-09-17T03:15:10.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.375+0000 7ff8b17fa640 1 -- 172.21.15.87:0/3915030405 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff8b8021070 con 0x7ff8bc1e0120 2024-09-17T03:15:10.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.375+0000 7ff8b17fa640 1 --2- 172.21.15.87:0/3915030405 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff8a8078b90 0x7ff8a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:10.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.376+0000 7ff8b17fa640 1 -- 172.21.15.87:0/3915030405 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7ff8b809be30 con 0x7ff8bc1e0120 2024-09-17T03:15:10.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.376+0000 7ff8b3fff640 1 --2- 172.21.15.87:0/3915030405 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff8a8078b90 0x7ff8a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:10.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.376+0000 7ff8b3fff640 1 --2- 172.21.15.87:0/3915030405 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff8a8078b90 0x7ff8a807b0a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac003670 tx=0x7ff8ac001ce0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:10.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.380+0000 7ff8b17fa640 1 -- 172.21.15.87:0/3915030405 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff8bc0b6a00 con 0x7ff8bc1e0120 2024-09-17T03:15:10.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.566+0000 7ff8c2ede640 1 -- 172.21.15.87:0/3915030405 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff8bc0b1de0 con 0x7ff8bc1e0120 2024-09-17T03:15:10.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.567+0000 7ff8b17fa640 1 -- 172.21.15.87:0/3915030405 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) ==== 80+0+0 (secure 0 0 0) 0x7ff8b8063bb0 con 0x7ff8bc1e0120 2024-09-17T03:15:10.566 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-09-17T03:15:10.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.569+0000 7ff8c2ede640 1 -- 172.21.15.87:0/3915030405 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff8a8078b90 msgr2=0x7ff8a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:10.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.569+0000 7ff8c2ede640 1 --2- 172.21.15.87:0/3915030405 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff8a8078b90 0x7ff8a807b0a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac003670 tx=0x7ff8ac001ce0 comp rx=0 tx=0).stop 2024-09-17T03:15:10.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.569+0000 7ff8c2ede640 1 -- 172.21.15.87:0/3915030405 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8bc1e0120 msgr2=0x7ff8bc1df7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:10.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.569+0000 7ff8c2ede640 1 --2- 172.21.15.87:0/3915030405 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8bc1e0120 0x7ff8bc1df7d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8011ba0 tx=0x7ff8b800f790 comp rx=0 tx=0).stop 2024-09-17T03:15:10.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.569+0000 7ff8c2ede640 1 -- 172.21.15.87:0/3915030405 shutdown_connections 2024-09-17T03:15:10.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.569+0000 7ff8c2ede640 1 --2- 172.21.15.87:0/3915030405 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff8a8078b90 0x7ff8a807b0a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:10.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.569+0000 7ff8c2ede640 1 --2- 172.21.15.87:0/3915030405 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff8bc0b65f0 0x7ff8bc1f3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:10.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.569+0000 7ff8c2ede640 1 --2- 172.21.15.87:0/3915030405 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff8bc1f6c40 0x7ff8bc0b60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:10.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.569+0000 7ff8c2ede640 1 --2- 172.21.15.87:0/3915030405 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff8bc1e0120 0x7ff8bc1df7d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:10.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.569+0000 7ff8c2ede640 1 -- 172.21.15.87:0/3915030405 >> 172.21.15.87:0/3915030405 conn(0x7ff8bc0d2230 msgr2=0x7ff8bc1e4910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:10.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.569+0000 7ff8c2ede640 1 -- 172.21.15.87:0/3915030405 shutdown_connections 2024-09-17T03:15:10.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:10.569+0000 7ff8c2ede640 1 -- 172.21.15.87:0/3915030405 wait complete. 2024-09-17T03:15:10.578 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi087.front.sepia.ceph.com... 2024-09-17T03:15:10.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:10.579 DEBUG:teuthology.orchestra.run.smithi087:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:10.601 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:ceph-fuse[49506]: fuse finished with error 0 and tester_r 0 2024-09-17T03:15:10.603 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:15:16.705 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-09-17T03:15:16.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:16.705 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T03:15:16.725 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T03:15:16.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.892+0000 7fbda20c5640 1 -- 172.21.15.87:0/3837145257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd9c0d0460 msgr2=0x7fbd9c0d0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:16.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.892+0000 7fbda20c5640 1 --2- 172.21.15.87:0/3837145257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd9c0d0460 0x7fbd9c0d0870 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbd94005fd0 tx=0x7fbd9402a820 comp rx=0 tx=0).stop 2024-09-17T03:15:16.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.892+0000 7fbda20c5640 1 -- 172.21.15.87:0/3837145257 shutdown_connections 2024-09-17T03:15:16.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.892+0000 7fbda20c5640 1 --2- 172.21.15.87:0/3837145257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd9c0d0460 0x7fbd9c0d0870 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:16.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.892+0000 7fbda20c5640 1 --2- 172.21.15.87:0/3837145257 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd9c15c6c0 0x7fbd9c160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:16.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.892+0000 7fbda20c5640 1 -- 172.21.15.87:0/3837145257 >> 172.21.15.87:0/3837145257 conn(0x7fbd9c069360 msgr2=0x7fbd9c0ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:16.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.892+0000 7fbda20c5640 1 -- 172.21.15.87:0/3837145257 shutdown_connections 2024-09-17T03:15:16.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.892+0000 7fbda20c5640 1 -- 172.21.15.87:0/3837145257 wait complete. 2024-09-17T03:15:16.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.894+0000 7fbda20c5640 1 Processor -- start 2024-09-17T03:15:16.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.894+0000 7fbda20c5640 1 -- start start 2024-09-17T03:15:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.895+0000 7fbda20c5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd9c0b3d00 0x7fbd9c1f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.895+0000 7fbda20c5640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd9c0d0460 0x7fbd9c1488a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.895+0000 7fbda20c5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd9c15c6c0 0x7fbd9c1f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.895+0000 7fbda20c5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c0d0f30 con 0x7fbd9c0b3d00 2024-09-17T03:15:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.895+0000 7fbda20c5640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c0d0db0 con 0x7fbd9c15c6c0 2024-09-17T03:15:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.895+0000 7fbda20c5640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbd9c0d10b0 con 0x7fbd9c0d0460 2024-09-17T03:15:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.895+0000 7fbd9b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd9c0b3d00 0x7fbd9c1f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.895+0000 7fbd9b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd9c0b3d00 0x7fbd9c1f3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34804/0 (socket says 172.21.15.87:34804) 2024-09-17T03:15:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.895+0000 7fbd9b7fe640 1 -- 172.21.15.87:0/3631387650 learned_addr learned my addr 172.21.15.87:0/3631387650 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.895+0000 7fbd9affd640 1 --2- 172.21.15.87:0/3631387650 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd9c0d0460 0x7fbd9c1488a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.895+0000 7fbd9bfff640 1 --2- 172.21.15.87:0/3631387650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd9c15c6c0 0x7fbd9c1f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.895+0000 7fbd9b7fe640 1 -- 172.21.15.87:0/3631387650 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd9c0d0460 msgr2=0x7fbd9c1488a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:16.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.896+0000 7fbd9b7fe640 1 --2- 172.21.15.87:0/3631387650 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd9c0d0460 0x7fbd9c1488a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:16.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.896+0000 7fbd9b7fe640 1 -- 172.21.15.87:0/3631387650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd9c15c6c0 msgr2=0x7fbd9c1f66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:16.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.896+0000 7fbd9b7fe640 1 --2- 172.21.15.87:0/3631387650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd9c15c6c0 0x7fbd9c1f66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:16.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.896+0000 7fbd9b7fe640 1 -- 172.21.15.87:0/3631387650 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9c15fde0 con 0x7fbd9c0b3d00 2024-09-17T03:15:16.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.896+0000 7fbd9bfff640 1 --2- 172.21.15.87:0/3631387650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd9c15c6c0 0x7fbd9c1f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:16.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.896+0000 7fbd9b7fe640 1 --2- 172.21.15.87:0/3631387650 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd9c0b3d00 0x7fbd9c1f3b30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c00d740 tx=0x7fbd8c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:16.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.896+0000 7fbd83fff640 1 -- 172.21.15.87:0/3631387650 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c017070 con 0x7fbd9c0b3d00 2024-09-17T03:15:16.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.896+0000 7fbd83fff640 1 -- 172.21.15.87:0/3631387650 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd8c009d20 con 0x7fbd9c0b3d00 2024-09-17T03:15:16.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.896+0000 7fbda20c5640 1 -- 172.21.15.87:0/3631387650 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9c15d1d0 con 0x7fbd9c0b3d00 2024-09-17T03:15:16.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.896+0000 7fbd83fff640 1 -- 172.21.15.87:0/3631387650 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c0106c0 con 0x7fbd9c0b3d00 2024-09-17T03:15:16.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.896+0000 7fbda20c5640 1 -- 172.21.15.87:0/3631387650 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd9c15d470 con 0x7fbd9c0b3d00 2024-09-17T03:15:16.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.897+0000 7fbda20c5640 1 -- 172.21.15.87:0/3631387650 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd9c0d1640 con 0x7fbd9c0b3d00 2024-09-17T03:15:16.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.898+0000 7fbd83fff640 1 -- 172.21.15.87:0/3631387650 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbd8c01a070 con 0x7fbd9c0b3d00 2024-09-17T03:15:16.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.898+0000 7fbd83fff640 1 --2- 172.21.15.87:0/3631387650 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbd88078bc0 0x7fbd8807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:16.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.898+0000 7fbd83fff640 1 -- 172.21.15.87:0/3631387650 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7fbd8c09b000 con 0x7fbd9c0b3d00 2024-09-17T03:15:16.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.898+0000 7fbd9affd640 1 --2- 172.21.15.87:0/3631387650 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbd88078bc0 0x7fbd8807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:16.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.899+0000 7fbd9affd640 1 --2- 172.21.15.87:0/3631387650 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbd88078bc0 0x7fbd8807b0d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0aa8f0 tx=0x7fbd90007e90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:16.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:16.903+0000 7fbd83fff640 1 -- 172.21.15.87:0/3631387650 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fbd8c062c70 con 0x7fbd9c0b3d00 2024-09-17T03:15:17.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:17.112+0000 7fbda20c5640 1 -- 172.21.15.87:0/3631387650 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbd9c0d2f00 con 0x7fbd9c0b3d00 2024-09-17T03:15:17.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:17.112+0000 7fbd83fff640 1 -- 172.21.15.87:0/3631387650 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+82 (secure 0 0 0) 0x7fbd8c0676a0 con 0x7fbd9c0b3d00 2024-09-17T03:15:17.111 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data data_new ] 2024-09-17T03:15:17.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:17.115+0000 7fbda20c5640 1 -- 172.21.15.87:0/3631387650 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbd88078bc0 msgr2=0x7fbd8807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:17.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:17.115+0000 7fbda20c5640 1 --2- 172.21.15.87:0/3631387650 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbd88078bc0 0x7fbd8807b0d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0aa8f0 tx=0x7fbd90007e90 comp rx=0 tx=0).stop 2024-09-17T03:15:17.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:17.115+0000 7fbda20c5640 1 -- 172.21.15.87:0/3631387650 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd9c0b3d00 msgr2=0x7fbd9c1f3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:17.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:17.115+0000 7fbda20c5640 1 --2- 172.21.15.87:0/3631387650 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd9c0b3d00 0x7fbd9c1f3b30 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c00d740 tx=0x7fbd8c00dc00 comp rx=0 tx=0).stop 2024-09-17T03:15:17.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:17.115+0000 7fbda20c5640 1 -- 172.21.15.87:0/3631387650 shutdown_connections 2024-09-17T03:15:17.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:17.115+0000 7fbda20c5640 1 --2- 172.21.15.87:0/3631387650 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbd88078bc0 0x7fbd8807b0d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:17.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:17.115+0000 7fbda20c5640 1 --2- 172.21.15.87:0/3631387650 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd9c15c6c0 0x7fbd9c1f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:17.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:17.115+0000 7fbda20c5640 1 --2- 172.21.15.87:0/3631387650 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd9c0d0460 0x7fbd9c1488a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:17.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:17.115+0000 7fbda20c5640 1 --2- 172.21.15.87:0/3631387650 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd9c0b3d00 0x7fbd9c1f3b30 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:17.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:17.115+0000 7fbda20c5640 1 -- 172.21.15.87:0/3631387650 >> 172.21.15.87:0/3631387650 conn(0x7fbd9c069360 msgr2=0x7fbd9c0d2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:17.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:17.115+0000 7fbda20c5640 1 -- 172.21.15.87:0/3631387650 shutdown_connections 2024-09-17T03:15:17.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:17.115+0000 7fbda20c5640 1 -- 172.21.15.87:0/3631387650 wait complete. 2024-09-17T03:15:17.125 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T03:15:17.125 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T03:15:17.125 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-09-17T03:15:17.125 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T03:15:17.125 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:17.125 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T03:15:17.125 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T03:15:17.125 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T03:15:17.126 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:15:17.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:17.126 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-09-17T03:15:17.141 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:15:17.142 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:15:17.142 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:15:17.142 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:17.142 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-09-17T03:15:17.142 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:17.142 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:15:17.142 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-09-17T03:15:17.142 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-09-17T03:15:17.142 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:15:17.142 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-09-17T03:15:17.142 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-09-17T03:15:17.142 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:15:17.143 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:15:17.143 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T03:15:17.143 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:15:17.143 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T03:15:17.143 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42715sec preferred_lft 42715sec 2024-09-17T03:15:17.143 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link noprefixroute 2024-09-17T03:15:17.143 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:17.143 INFO:teuthology.orchestra.run.smithi087.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:15:17.143 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 26:32:f7:da:05:b7 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:15:17.143 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T03:15:17.143 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:17.143 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c822:ceff:fef0:4a99/64 scope link 2024-09-17T03:15:17.143 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:17.143 INFO:teuthology.orchestra.run.smithi087.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:15:17.144 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether b2:cb:c1:94:14:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:15:17.144 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::b0cb:c1ff:fe94:145d/64 scope link 2024-09-17T03:15:17.144 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:17.144 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:15:17.144 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 26:32:f7:da:05:b7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:15:17.144 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::2432:f7ff:feda:5b7/64 scope link 2024-09-17T03:15:17.144 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:17.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:17.144 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T03:15:17.205 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-09-17T03:15:17.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:17.207 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-09-17T03:15:17.262 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-09-17T03:15:17.262 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T03:15:17.263 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T03:15:17.263 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T03:15:17.263 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T03:15:17.263 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:15:17.263 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-09-17T03:15:17.263 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T03:15:17.263 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T03:15:17.263 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T03:15:17.263 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-09-17T03:15:17.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:15:17.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:15:17.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:17.388 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-09-17T03:15:17.403 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T03:15:17.404 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T03:15:17.404 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:15:17.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:17.404 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-09-17T03:15:17.468 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T03:15:17.468 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:15:17.469 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:17.524 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:15:17.524 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:15:17.524 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:17.579 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-09-17T03:15:17.645 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T03:15:17.702 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-09-17T03:15:17.702 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-09-17T03:15:17.702 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-09-17T03:15:17.702 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-09-17T03:15:17.702 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-09-17T03:15:17.702 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-09-17T03:15:17.702 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-09-17T03:15:17.702 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-09-17T03:15:17.702 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-09-17T03:15:17.702 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-09-17T03:15:17.702 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-09-17T03:15:17.702 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-09-17T03:15:17.703 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T03:15:17.703 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-09-17T03:15:17.703 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-09-17T03:15:17.703 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-09-17T03:15:17.703 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-09-17T03:15:17.703 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-09-17T03:15:17.703 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-09-17T03:15:17.703 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T03:15:17.703 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T03:15:17.703 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T03:15:17.703 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T03:15:17.703 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-09-17T03:15:17.703 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-09-17T03:15:17.704 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T03:15:17.704 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T03:15:17.704 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T03:15:17.704 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T03:15:17.704 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-09-17T03:15:17.704 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:15:17.704 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:15:17.704 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:15:17.704 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:15:17.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:17.704 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-09-17T03:15:17.757 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T03:15:17.757 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-17T03:15:17.800 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-09-17T03:15:17.824 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T03:15:17.881 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-09-17T03:15:17.881 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-09-17T03:15:17.882 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-09-17T03:15:17.882 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-09-17T03:15:17.882 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-09-17T03:15:17.882 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-09-17T03:15:17.882 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-09-17T03:15:17.882 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-09-17T03:15:17.882 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-09-17T03:15:17.882 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-09-17T03:15:17.882 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-09-17T03:15:17.882 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-09-17T03:15:17.882 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T03:15:17.882 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-09-17T03:15:17.882 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-09-17T03:15:17.883 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-09-17T03:15:17.883 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-09-17T03:15:17.883 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-09-17T03:15:17.883 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-09-17T03:15:17.883 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T03:15:17.883 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T03:15:17.883 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T03:15:17.883 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T03:15:17.883 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-09-17T03:15:17.883 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-09-17T03:15:17.883 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T03:15:17.883 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T03:15:17.883 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T03:15:17.884 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T03:15:17.884 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-09-17T03:15:17.884 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:15:17.884 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:15:17.884 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:15:17.884 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:15:17.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:17.884 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-09-17T03:15:17.892 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:2024-09-17T03:15:17.892+0000 7fe1f340a480 -1 init, newargv = 0x7fe1d4004d30 newargc=15 2024-09-17T03:15:17.892 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:ceph-fuse[50339]: starting ceph client 2024-09-17T03:15:17.896 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:ceph-fuse[50339]: starting fuse 2024-09-17T03:15:17.906 INFO:teuthology.orchestra.run.smithi087.stdout:42 2024-09-17T03:15:17.907 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-17T03:15:17.907 DEBUG:teuthology.orchestra.run.smithi087:> sudo stdin-killer -- python3 -c ' 2024-09-17T03:15:17.907 DEBUG:teuthology.orchestra.run.smithi087:> import glob 2024-09-17T03:15:17.907 DEBUG:teuthology.orchestra.run.smithi087:> import re 2024-09-17T03:15:17.907 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-09-17T03:15:17.907 DEBUG:teuthology.orchestra.run.smithi087:> import subprocess 2024-09-17T03:15:17.907 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:15:17.907 DEBUG:teuthology.orchestra.run.smithi087:> def _find_admin_socket(client_name): 2024-09-17T03:15:17.907 DEBUG:teuthology.orchestra.run.smithi087:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T03:15:17.907 DEBUG:teuthology.orchestra.run.smithi087:> files = glob.glob(asok_path) 2024-09-17T03:15:17.908 DEBUG:teuthology.orchestra.run.smithi087:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T03:15:17.908 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:15:17.908 DEBUG:teuthology.orchestra.run.smithi087:> # Given a non-glob path, it better be there 2024-09-17T03:15:17.908 DEBUG:teuthology.orchestra.run.smithi087:> if "*" not in asok_path: 2024-09-17T03:15:17.908 DEBUG:teuthology.orchestra.run.smithi087:> assert(len(files) == 1) 2024-09-17T03:15:17.908 DEBUG:teuthology.orchestra.run.smithi087:> return files[0] 2024-09-17T03:15:17.908 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:15:17.908 DEBUG:teuthology.orchestra.run.smithi087:> for f in files: 2024-09-17T03:15:17.908 DEBUG:teuthology.orchestra.run.smithi087:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T03:15:17.908 DEBUG:teuthology.orchestra.run.smithi087:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T03:15:17.908 DEBUG:teuthology.orchestra.run.smithi087:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T03:15:17.908 DEBUG:teuthology.orchestra.run.smithi087:> contents = proc_f.read() 2024-09-17T03:15:17.908 DEBUG:teuthology.orchestra.run.smithi087:> if mountpoint in contents: 2024-09-17T03:15:17.908 DEBUG:teuthology.orchestra.run.smithi087:> return f 2024-09-17T03:15:17.909 DEBUG:teuthology.orchestra.run.smithi087:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T03:15:17.909 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:15:17.909 DEBUG:teuthology.orchestra.run.smithi087:> print(_find_admin_socket("client.0")) 2024-09-17T03:15:17.909 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-09-17T03:15:18.023 INFO:teuthology.orchestra.run.smithi087.stdout:/var/run/ceph/ceph-client.0.50339.asok 2024-09-17T03:15:18.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:15:18.033 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50339.asok 2024-09-17T03:15:18.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:18.033 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50339.asok status 2024-09-17T03:15:18.175 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-17T03:15:18.175 INFO:teuthology.orchestra.run.smithi087.stdout: "metadata": { 2024-09-17T03:15:18.175 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T03:15:18.175 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T03:15:18.175 INFO:teuthology.orchestra.run.smithi087.stdout: "entity_id": "0", 2024-09-17T03:15:18.175 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": "smithi087", 2024-09-17T03:15:18.175 INFO:teuthology.orchestra.run.smithi087.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T03:15:18.175 INFO:teuthology.orchestra.run.smithi087.stdout: "pid": "50339", 2024-09-17T03:15:18.175 INFO:teuthology.orchestra.run.smithi087.stdout: "root": "/" 2024-09-17T03:15:18.175 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:18.176 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_count": 0, 2024-09-17T03:15:18.176 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_pinned_count": 0, 2024-09-17T03:15:18.176 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 5274, 2024-09-17T03:15:18.176 INFO:teuthology.orchestra.run.smithi087.stdout: "inst": { 2024-09-17T03:15:18.176 INFO:teuthology.orchestra.run.smithi087.stdout: "name": { 2024-09-17T03:15:18.176 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "client", 2024-09-17T03:15:18.176 INFO:teuthology.orchestra.run.smithi087.stdout: "num": 5274 2024-09-17T03:15:18.176 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:18.176 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-09-17T03:15:18.176 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-09-17T03:15:18.176 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-09-17T03:15:18.176 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 109486574 2024-09-17T03:15:18.176 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:18.177 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:18.177 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-09-17T03:15:18.177 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-09-17T03:15:18.177 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-09-17T03:15:18.177 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 109486574 2024-09-17T03:15:18.177 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:18.177 INFO:teuthology.orchestra.run.smithi087.stdout: "inst_str": "client.5274 192.168.0.1:0/109486574", 2024-09-17T03:15:18.177 INFO:teuthology.orchestra.run.smithi087.stdout: "addr_str": "192.168.0.1:0/109486574", 2024-09-17T03:15:18.177 INFO:teuthology.orchestra.run.smithi087.stdout: "inode_count": 1, 2024-09-17T03:15:18.177 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_epoch": 14, 2024-09-17T03:15:18.177 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch": 37, 2024-09-17T03:15:18.177 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch_barrier": 0, 2024-09-17T03:15:18.177 INFO:teuthology.orchestra.run.smithi087.stdout: "blocklisted": false, 2024-09-17T03:15:18.177 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs" 2024-09-17T03:15:18.178 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-17T03:15:18.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:18.184 DEBUG:teuthology.orchestra.run.smithi087:> stat --file-system '--printf=%T 2024-09-17T03:15:18.184 DEBUG:teuthology.orchestra.run.smithi087:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:18.200 INFO:teuthology.orchestra.run.smithi087.stdout:fuseblk 2024-09-17T03:15:18.201 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:18.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:18.201 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:18.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:18.266 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2024-09-17T03:15:18.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:15:18.326 INFO:teuthology.orchestra.run.smithi087.stderr:setfattr: layoutfile: Permission denied 2024-09-17T03:15:18.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:18.327 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2024-09-17T03:15:18.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:15:18.383 INFO:teuthology.orchestra.run.smithi087.stderr:setfattr: layoutdir: Permission denied 2024-09-17T03:15:18.384 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:15:18.384 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:15:18.440 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:15:18.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.594+0000 7f3118912640 1 -- 172.21.15.87:0/2784797971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3114072530 msgr2=0x7f311415f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:18.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.594+0000 7f3118912640 1 --2- 172.21.15.87:0/2784797971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3114072530 0x7f311415f510 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f31140738a0 tx=0x7f30fc02a5f0 comp rx=0 tx=0).stop 2024-09-17T03:15:18.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.594+0000 7f3118912640 1 -- 172.21.15.87:0/2784797971 shutdown_connections 2024-09-17T03:15:18.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.594+0000 7f3118912640 1 --2- 172.21.15.87:0/2784797971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3114072530 0x7f311415f510 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:18.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.594+0000 7f3118912640 1 --2- 172.21.15.87:0/2784797971 >> v2:172.21.15.149:3300/0 conn(0x7f3114072120 0x7f311415efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:18.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.594+0000 7f3118912640 1 -- 172.21.15.87:0/2784797971 >> 172.21.15.87:0/2784797971 conn(0x7f31140d1180 msgr2=0x7f31140d1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:18.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.594+0000 7f3118912640 1 -- 172.21.15.87:0/2784797971 shutdown_connections 2024-09-17T03:15:18.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.594+0000 7f3118912640 1 -- 172.21.15.87:0/2784797971 wait complete. 2024-09-17T03:15:18.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.596+0000 7f3118912640 1 Processor -- start 2024-09-17T03:15:18.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.596+0000 7f3118912640 1 -- start start 2024-09-17T03:15:18.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.597+0000 7f3118912640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3114072120 0x7f311414e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:18.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.597+0000 7f3118912640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3114072530 0x7f3114159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:18.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.597+0000 7f3118912640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31140b4110 0x7f311415a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:18.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.597+0000 7f3118912640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3114074210 con 0x7f3114072120 2024-09-17T03:15:18.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.597+0000 7f3118912640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3114074090 con 0x7f31140b4110 2024-09-17T03:15:18.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.597+0000 7f3118912640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3114074390 con 0x7f3114072530 2024-09-17T03:15:18.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.597+0000 7f311259c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3114072120 0x7f311414e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:18.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.597+0000 7f311259c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3114072120 0x7f311414e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34820/0 (socket says 172.21.15.87:34820) 2024-09-17T03:15:18.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.598+0000 7f311259c640 1 -- 172.21.15.87:0/3418574646 learned_addr learned my addr 172.21.15.87:0/3418574646 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:18.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.598+0000 7f3112d9d640 1 --2- 172.21.15.87:0/3418574646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31140b4110 0x7f311415a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:18.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.598+0000 7f3109d9b640 1 --2- 172.21.15.87:0/3418574646 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3114072530 0x7f3114159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:18.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.598+0000 7f311259c640 1 -- 172.21.15.87:0/3418574646 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3114072530 msgr2=0x7f3114159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:18.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.598+0000 7f311259c640 1 --2- 172.21.15.87:0/3418574646 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3114072530 0x7f3114159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:18.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.598+0000 7f311259c640 1 -- 172.21.15.87:0/3418574646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31140b4110 msgr2=0x7f311415a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:18.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.598+0000 7f311259c640 1 --2- 172.21.15.87:0/3418574646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31140b4110 0x7f311415a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:18.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.598+0000 7f311259c640 1 -- 172.21.15.87:0/3418574646 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31141f5b30 con 0x7f3114072120 2024-09-17T03:15:18.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.598+0000 7f311259c640 1 --2- 172.21.15.87:0/3418574646 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3114072120 0x7f311414e950 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f310400d700 tx=0x7f310400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:18.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.600+0000 7f310b7fe640 1 -- 172.21.15.87:0/3418574646 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3104017070 con 0x7f3114072120 2024-09-17T03:15:18.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.600+0000 7f310b7fe640 1 -- 172.21.15.87:0/3418574646 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3104005310 con 0x7f3114072120 2024-09-17T03:15:18.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.600+0000 7f3118912640 1 -- 172.21.15.87:0/3418574646 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31141f5e10 con 0x7f3114072120 2024-09-17T03:15:18.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.600+0000 7f310b7fe640 1 -- 172.21.15.87:0/3418574646 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31040097c0 con 0x7f3114072120 2024-09-17T03:15:18.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.600+0000 7f3118912640 1 -- 172.21.15.87:0/3418574646 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31141f63d0 con 0x7f3114072120 2024-09-17T03:15:18.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.601+0000 7f3118912640 1 -- 172.21.15.87:0/3418574646 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3114074840 con 0x7f3114072120 2024-09-17T03:15:18.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.601+0000 7f310b7fe640 1 -- 172.21.15.87:0/3418574646 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3104009960 con 0x7f3114072120 2024-09-17T03:15:18.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.602+0000 7f310b7fe640 1 --2- 172.21.15.87:0/3418574646 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f30f4078bc0 0x7f30f407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:18.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.602+0000 7f310b7fe640 1 -- 172.21.15.87:0/3418574646 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7f310409ad90 con 0x7f3114072120 2024-09-17T03:15:18.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.602+0000 7f3109d9b640 1 --2- 172.21.15.87:0/3418574646 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f30f4078bc0 0x7f30f407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:18.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.603+0000 7f3109d9b640 1 --2- 172.21.15.87:0/3418574646 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f30f4078bc0 0x7f30f407b0d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f30fc033000 tx=0x7f30fc004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:18.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.607+0000 7f310b7fe640 1 -- 172.21.15.87:0/3418574646 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3104062b10 con 0x7f3114072120 2024-09-17T03:15:18.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.791+0000 7f3118912640 1 -- 172.21.15.87:0/3418574646 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f311415fe50 con 0x7f3114072120 2024-09-17T03:15:18.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.791+0000 7f310b7fe640 1 -- 172.21.15.87:0/3418574646 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) ==== 80+0+0 (secure 0 0 0) 0x7f3104067540 con 0x7f3114072120 2024-09-17T03:15:18.790 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-09-17T03:15:18.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.793+0000 7f3118912640 1 -- 172.21.15.87:0/3418574646 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f30f4078bc0 msgr2=0x7f30f407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:18.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.793+0000 7f3118912640 1 --2- 172.21.15.87:0/3418574646 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f30f4078bc0 0x7f30f407b0d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f30fc033000 tx=0x7f30fc004730 comp rx=0 tx=0).stop 2024-09-17T03:15:18.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.793+0000 7f3118912640 1 -- 172.21.15.87:0/3418574646 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3114072120 msgr2=0x7f311414e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:18.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.793+0000 7f3118912640 1 --2- 172.21.15.87:0/3418574646 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3114072120 0x7f311414e950 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f310400d700 tx=0x7f310400dbc0 comp rx=0 tx=0).stop 2024-09-17T03:15:18.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.793+0000 7f3118912640 1 -- 172.21.15.87:0/3418574646 shutdown_connections 2024-09-17T03:15:18.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.793+0000 7f3118912640 1 --2- 172.21.15.87:0/3418574646 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f30f4078bc0 0x7f30f407b0d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:18.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.793+0000 7f3118912640 1 --2- 172.21.15.87:0/3418574646 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31140b4110 0x7f311415a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:18.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.793+0000 7f3118912640 1 --2- 172.21.15.87:0/3418574646 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3114072530 0x7f3114159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:18.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.793+0000 7f3118912640 1 --2- 172.21.15.87:0/3418574646 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3114072120 0x7f311414e950 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:18.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.793+0000 7f3118912640 1 -- 172.21.15.87:0/3418574646 >> 172.21.15.87:0/3418574646 conn(0x7f31140d1180 msgr2=0x7f31140cd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:18.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.793+0000 7f3118912640 1 -- 172.21.15.87:0/3418574646 shutdown_connections 2024-09-17T03:15:18.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.794+0000 7f3118912640 1 -- 172.21.15.87:0/3418574646 wait complete. 2024-09-17T03:15:18.803 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:15:18.803 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:15:18.819 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:15:18.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.974+0000 7f9a5c1df640 1 -- 172.21.15.87:0/3066376219 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a540b5ef0 msgr2=0x7f9a540b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:18.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.974+0000 7f9a5c1df640 1 --2- 172.21.15.87:0/3066376219 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a540b5ef0 0x7f9a540b6300 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c009830 tx=0x7f9a4c02a500 comp rx=0 tx=0).stop 2024-09-17T03:15:18.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.974+0000 7f9a5c1df640 1 -- 172.21.15.87:0/3066376219 shutdown_connections 2024-09-17T03:15:18.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.974+0000 7f9a5c1df640 1 --2- 172.21.15.87:0/3066376219 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a540b5ef0 0x7f9a540b6300 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:18.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.974+0000 7f9a5c1df640 1 --2- 172.21.15.87:0/3066376219 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9a5415ecf0 0x7f9a5415f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:18.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.974+0000 7f9a5c1df640 1 -- 172.21.15.87:0/3066376219 >> 172.21.15.87:0/3066376219 conn(0x7f9a540d4470 msgr2=0x7f9a540d4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:18.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.974+0000 7f9a5c1df640 1 -- 172.21.15.87:0/3066376219 shutdown_connections 2024-09-17T03:15:18.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.974+0000 7f9a5c1df640 1 -- 172.21.15.87:0/3066376219 wait complete. 2024-09-17T03:15:18.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.975+0000 7f9a5c1df640 1 Processor -- start 2024-09-17T03:15:18.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.976+0000 7f9a5c1df640 1 -- start start 2024-09-17T03:15:18.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.977+0000 7f9a5c1df640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a540b5ef0 0x7f9a5414ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:18.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.977+0000 7f9a5c1df640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a5415c2d0 0x7f9a540c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:18.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.977+0000 7f9a5c1df640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9a5415ecf0 0x7f9a5414c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:18.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.977+0000 7f9a5c1df640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9a540d2db0 con 0x7f9a5415c2d0 2024-09-17T03:15:18.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.977+0000 7f9a5c1df640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9a540d2c30 con 0x7f9a540b5ef0 2024-09-17T03:15:18.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.977+0000 7f9a5c1df640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9a540d2f30 con 0x7f9a5415ecf0 2024-09-17T03:15:18.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.977+0000 7f9a5977a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a5415c2d0 0x7f9a540c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:18.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.977+0000 7f9a5977a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a5415c2d0 0x7f9a540c9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34840/0 (socket says 172.21.15.87:34840) 2024-09-17T03:15:18.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.977+0000 7f9a5977a640 1 -- 172.21.15.87:0/800215870 learned_addr learned my addr 172.21.15.87:0/800215870 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:18.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.977+0000 7f9a59f7b640 1 --2- 172.21.15.87:0/800215870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a540b5ef0 0x7f9a5414ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:18.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.977+0000 7f9a5a77c640 1 --2- 172.21.15.87:0/800215870 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9a5415ecf0 0x7f9a5414c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:18.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.977+0000 7f9a5977a640 1 -- 172.21.15.87:0/800215870 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9a5415ecf0 msgr2=0x7f9a5414c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:18.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.977+0000 7f9a5977a640 1 --2- 172.21.15.87:0/800215870 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9a5415ecf0 0x7f9a5414c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:18.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.977+0000 7f9a5977a640 1 -- 172.21.15.87:0/800215870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a540b5ef0 msgr2=0x7f9a5414ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:18.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.977+0000 7f9a5977a640 1 --2- 172.21.15.87:0/800215870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a540b5ef0 0x7f9a5414ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:18.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.977+0000 7f9a5977a640 1 -- 172.21.15.87:0/800215870 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a540c8850 con 0x7f9a5415c2d0 2024-09-17T03:15:18.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.977+0000 7f9a5977a640 1 --2- 172.21.15.87:0/800215870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a5415c2d0 0x7f9a540c9df0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9a4400d740 tx=0x7f9a4400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:18.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.978+0000 7f9a4a7fc640 1 -- 172.21.15.87:0/800215870 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a44017070 con 0x7f9a5415c2d0 2024-09-17T03:15:18.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.978+0000 7f9a4a7fc640 1 -- 172.21.15.87:0/800215870 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a44009d20 con 0x7f9a5415c2d0 2024-09-17T03:15:18.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.978+0000 7f9a5c1df640 1 -- 172.21.15.87:0/800215870 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a540c8d50 con 0x7f9a5415c2d0 2024-09-17T03:15:18.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.978+0000 7f9a4a7fc640 1 -- 172.21.15.87:0/800215870 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a440106c0 con 0x7f9a5415c2d0 2024-09-17T03:15:18.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.978+0000 7f9a5c1df640 1 -- 172.21.15.87:0/800215870 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a540c9310 con 0x7f9a5415c2d0 2024-09-17T03:15:18.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.979+0000 7f9a5c1df640 1 -- 172.21.15.87:0/800215870 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a540c8ac0 con 0x7f9a5415c2d0 2024-09-17T03:15:18.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.979+0000 7f9a4a7fc640 1 -- 172.21.15.87:0/800215870 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9a4401a070 con 0x7f9a5415c2d0 2024-09-17T03:15:18.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.980+0000 7f9a4a7fc640 1 --2- 172.21.15.87:0/800215870 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9a38078b90 0x7f9a3807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:18.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.980+0000 7f9a4a7fc640 1 -- 172.21.15.87:0/800215870 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7f9a4409af60 con 0x7f9a5415c2d0 2024-09-17T03:15:18.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.980+0000 7f9a59f7b640 1 --2- 172.21.15.87:0/800215870 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9a38078b90 0x7f9a3807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:18.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.981+0000 7f9a59f7b640 1 --2- 172.21.15.87:0/800215870 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9a38078b90 0x7f9a3807b0a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9a50002200 tx=0x7f9a500019f0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:18.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:18.984+0000 7f9a4a7fc640 1 -- 172.21.15.87:0/800215870 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9a44062c50 con 0x7f9a5415c2d0 2024-09-17T03:15:19.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.169+0000 7f9a5c1df640 1 -- 172.21.15.87:0/800215870 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9a540d3400 con 0x7f9a5415c2d0 2024-09-17T03:15:19.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.169+0000 7f9a4a7fc640 1 -- 172.21.15.87:0/800215870 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) ==== 80+0+0 (secure 0 0 0) 0x7f9a44067680 con 0x7f9a5415c2d0 2024-09-17T03:15:19.168 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-09-17T03:15:19.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.171+0000 7f9a5c1df640 1 -- 172.21.15.87:0/800215870 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9a38078b90 msgr2=0x7f9a3807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:19.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.171+0000 7f9a5c1df640 1 --2- 172.21.15.87:0/800215870 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9a38078b90 0x7f9a3807b0a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9a50002200 tx=0x7f9a500019f0 comp rx=0 tx=0).stop 2024-09-17T03:15:19.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.171+0000 7f9a5c1df640 1 -- 172.21.15.87:0/800215870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a5415c2d0 msgr2=0x7f9a540c9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:19.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.171+0000 7f9a5c1df640 1 --2- 172.21.15.87:0/800215870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a5415c2d0 0x7f9a540c9df0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9a4400d740 tx=0x7f9a4400dc00 comp rx=0 tx=0).stop 2024-09-17T03:15:19.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.171+0000 7f9a5c1df640 1 -- 172.21.15.87:0/800215870 shutdown_connections 2024-09-17T03:15:19.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.171+0000 7f9a5c1df640 1 --2- 172.21.15.87:0/800215870 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9a38078b90 0x7f9a3807b0a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:19.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.171+0000 7f9a5c1df640 1 --2- 172.21.15.87:0/800215870 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9a5415ecf0 0x7f9a5414c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:19.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.171+0000 7f9a5c1df640 1 --2- 172.21.15.87:0/800215870 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a5415c2d0 0x7f9a540c9df0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:19.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.171+0000 7f9a5c1df640 1 --2- 172.21.15.87:0/800215870 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9a540b5ef0 0x7f9a5414ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:19.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.171+0000 7f9a5c1df640 1 -- 172.21.15.87:0/800215870 >> 172.21.15.87:0/800215870 conn(0x7f9a540d4470 msgr2=0x7f9a5415fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:19.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.171+0000 7f9a5c1df640 1 -- 172.21.15.87:0/800215870 shutdown_connections 2024-09-17T03:15:19.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.171+0000 7f9a5c1df640 1 -- 172.21.15.87:0/800215870 wait complete. 2024-09-17T03:15:19.181 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi087.front.sepia.ceph.com... 2024-09-17T03:15:19.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:19.181 DEBUG:teuthology.orchestra.run.smithi087:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:19.204 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:ceph-fuse[50339]: fuse finished with error 0 and tester_r 0 2024-09-17T03:15:19.206 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:15:19.269 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-09-17T03:15:19.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:19.270 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T03:15:19.285 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds 'allow rwp' mon 'allow r' osd 'allow rw pool=cephfs_data,allow rw pool=data_new' 2024-09-17T03:15:19.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.446+0000 7fbd864ce640 1 -- 172.21.15.87:0/135020198 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd8015c5d0 msgr2=0x7fbd80160c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:19.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.446+0000 7fbd864ce640 1 --2- 172.21.15.87:0/135020198 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd8015c5d0 0x7fbd80160c50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbd800cd640 tx=0x7fbd6802a530 comp rx=0 tx=0).stop 2024-09-17T03:15:19.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.446+0000 7fbd864ce640 1 -- 172.21.15.87:0/135020198 shutdown_connections 2024-09-17T03:15:19.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.446+0000 7fbd864ce640 1 --2- 172.21.15.87:0/135020198 >> v2:172.21.15.149:3300/0 conn(0x7fbd800ccaf0 0x7fbd800ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:19.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.446+0000 7fbd864ce640 1 --2- 172.21.15.87:0/135020198 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd8015c5d0 0x7fbd80160c50 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:19.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.446+0000 7fbd864ce640 1 -- 172.21.15.87:0/135020198 >> 172.21.15.87:0/135020198 conn(0x7fbd80069120 msgr2=0x7fbd80069530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:19.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.446+0000 7fbd864ce640 1 -- 172.21.15.87:0/135020198 shutdown_connections 2024-09-17T03:15:19.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.446+0000 7fbd864ce640 1 -- 172.21.15.87:0/135020198 wait complete. 2024-09-17T03:15:19.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.447+0000 7fbd864ce640 1 Processor -- start 2024-09-17T03:15:19.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.448+0000 7fbd864ce640 1 -- start start 2024-09-17T03:15:19.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.449+0000 7fbd864ce640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd800ccaf0 0x7fbd80148440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:19.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.449+0000 7fbd864ce640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd800d1fe0 0x7fbd800cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:19.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.449+0000 7fbd864ce640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd8015c5d0 0x7fbd801f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:19.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.449+0000 7fbd864ce640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbd800d0a90 con 0x7fbd8015c5d0 2024-09-17T03:15:19.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.449+0000 7fbd864ce640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbd800d0910 con 0x7fbd800ccaf0 2024-09-17T03:15:19.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.449+0000 7fbd864ce640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbd800d0c10 con 0x7fbd800d1fe0 2024-09-17T03:15:19.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.449+0000 7fbd84a6b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd8015c5d0 0x7fbd801f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:19.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.449+0000 7fbd84a6b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd8015c5d0 0x7fbd801f6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34850/0 (socket says 172.21.15.87:34850) 2024-09-17T03:15:19.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.449+0000 7fbd84a6b640 1 -- 172.21.15.87:0/3032631925 learned_addr learned my addr 172.21.15.87:0/3032631925 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:19.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.449+0000 7fbd7ffff640 1 --2- 172.21.15.87:0/3032631925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd800ccaf0 0x7fbd80148440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:19.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.449+0000 7fbd7f7fe640 1 --2- 172.21.15.87:0/3032631925 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd800d1fe0 0x7fbd800cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:19.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.449+0000 7fbd84a6b640 1 -- 172.21.15.87:0/3032631925 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd800d1fe0 msgr2=0x7fbd800cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:19.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.449+0000 7fbd84a6b640 1 --2- 172.21.15.87:0/3032631925 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd800d1fe0 0x7fbd800cc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:19.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.449+0000 7fbd84a6b640 1 -- 172.21.15.87:0/3032631925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd800ccaf0 msgr2=0x7fbd80148440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:19.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.449+0000 7fbd84a6b640 1 --2- 172.21.15.87:0/3032631925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd800ccaf0 0x7fbd80148440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:19.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.449+0000 7fbd84a6b640 1 -- 172.21.15.87:0/3032631925 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd80148980 con 0x7fbd8015c5d0 2024-09-17T03:15:19.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.449+0000 7fbd7ffff640 1 --2- 172.21.15.87:0/3032631925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd800ccaf0 0x7fbd80148440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:15:19.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.450+0000 7fbd84a6b640 1 --2- 172.21.15.87:0/3032631925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd8015c5d0 0x7fbd801f6ce0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800e960 tx=0x7fbd7800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:19.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.450+0000 7fbd7cff9640 1 -- 172.21.15.87:0/3032631925 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd78004d20 con 0x7fbd8015c5d0 2024-09-17T03:15:19.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.450+0000 7fbd7cff9640 1 -- 172.21.15.87:0/3032631925 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd7800cb20 con 0x7fbd8015c5d0 2024-09-17T03:15:19.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.450+0000 7fbd7cff9640 1 -- 172.21.15.87:0/3032631925 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd78005020 con 0x7fbd8015c5d0 2024-09-17T03:15:19.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.450+0000 7fbd864ce640 1 -- 172.21.15.87:0/3032631925 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd8015ed20 con 0x7fbd8015c5d0 2024-09-17T03:15:19.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.450+0000 7fbd864ce640 1 -- 172.21.15.87:0/3032631925 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd8015f2e0 con 0x7fbd8015c5d0 2024-09-17T03:15:19.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.451+0000 7fbd864ce640 1 -- 172.21.15.87:0/3032631925 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd8015ea90 con 0x7fbd8015c5d0 2024-09-17T03:15:19.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.452+0000 7fbd7cff9640 1 -- 172.21.15.87:0/3032631925 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbd780051c0 con 0x7fbd8015c5d0 2024-09-17T03:15:19.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.452+0000 7fbd7cff9640 1 --2- 172.21.15.87:0/3032631925 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbd60078b90 0x7fbd6007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:19.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.452+0000 7fbd7cff9640 1 -- 172.21.15.87:0/3032631925 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7fbd7809b6f0 con 0x7fbd8015c5d0 2024-09-17T03:15:19.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.453+0000 7fbd7ffff640 1 --2- 172.21.15.87:0/3032631925 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbd60078b90 0x7fbd6007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:19.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.453+0000 7fbd7ffff640 1 --2- 172.21.15.87:0/3032631925 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbd60078b90 0x7fbd6007b0a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbd700013f0 tx=0x7fbd70009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:19.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.457+0000 7fbd7cff9640 1 -- 172.21.15.87:0/3032631925 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fbd78016030 con 0x7fbd8015c5d0 2024-09-17T03:15:19.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.702+0000 7fbd864ce640 1 -- 172.21.15.87:0/3032631925 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]} v 0) -- 0x7fbd800d1150 con 0x7fbd8015c5d0 2024-09-17T03:15:19.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.704+0000 7fbd7cff9640 1 -- 172.21.15.87:0/3032631925 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow rwp", "mon", "allow r", "osd", "allow rw pool=cephfs_data,allow rw pool=data_new"]}]=0 updated caps for client.0 v15) ==== 211+0+168 (secure 0 0 0) 0x7fbd78063ac0 con 0x7fbd8015c5d0 2024-09-17T03:15:19.704 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-09-17T03:15:19.704 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDH8+hmLskNLhAAd2M5ynGP8W6Pim11F18wYA== 2024-09-17T03:15:19.704 INFO:teuthology.orchestra.run.smithi087.stdout: caps mds = "allow rwp" 2024-09-17T03:15:19.704 INFO:teuthology.orchestra.run.smithi087.stdout: caps mon = "allow r" 2024-09-17T03:15:19.704 INFO:teuthology.orchestra.run.smithi087.stdout: caps osd = "allow rw pool=cephfs_data,allow rw pool=data_new" 2024-09-17T03:15:19.704 INFO:teuthology.orchestra.run.smithi087.stderr:updated caps for client.0 2024-09-17T03:15:19.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.706+0000 7fbd864ce640 1 -- 172.21.15.87:0/3032631925 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbd60078b90 msgr2=0x7fbd6007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:19.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.706+0000 7fbd864ce640 1 --2- 172.21.15.87:0/3032631925 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbd60078b90 0x7fbd6007b0a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fbd700013f0 tx=0x7fbd70009040 comp rx=0 tx=0).stop 2024-09-17T03:15:19.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.706+0000 7fbd864ce640 1 -- 172.21.15.87:0/3032631925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd8015c5d0 msgr2=0x7fbd801f6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:19.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.706+0000 7fbd864ce640 1 --2- 172.21.15.87:0/3032631925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd8015c5d0 0x7fbd801f6ce0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800e960 tx=0x7fbd7800ee20 comp rx=0 tx=0).stop 2024-09-17T03:15:19.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.706+0000 7fbd864ce640 1 -- 172.21.15.87:0/3032631925 shutdown_connections 2024-09-17T03:15:19.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.706+0000 7fbd864ce640 1 --2- 172.21.15.87:0/3032631925 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbd60078b90 0x7fbd6007b0a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:19.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.706+0000 7fbd864ce640 1 --2- 172.21.15.87:0/3032631925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd8015c5d0 0x7fbd801f6ce0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:19.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.706+0000 7fbd864ce640 1 --2- 172.21.15.87:0/3032631925 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd800d1fe0 0x7fbd800cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:19.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.706+0000 7fbd864ce640 1 --2- 172.21.15.87:0/3032631925 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd800ccaf0 0x7fbd80148440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:19.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.707+0000 7fbd864ce640 1 -- 172.21.15.87:0/3032631925 >> 172.21.15.87:0/3032631925 conn(0x7fbd80069120 msgr2=0x7fbd800b5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:19.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.707+0000 7fbd864ce640 1 -- 172.21.15.87:0/3032631925 shutdown_connections 2024-09-17T03:15:19.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.707+0000 7fbd864ce640 1 -- 172.21.15.87:0/3032631925 wait complete. 2024-09-17T03:15:19.717 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T03:15:19.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.829+0000 7f93cd0d3640 1 -- 172.21.15.87:0/1777957598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93c80b3d60 msgr2=0x7f93c80b4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:19.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.829+0000 7f93cd0d3640 1 --2- 172.21.15.87:0/1777957598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93c80b3d60 0x7f93c80b4170 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f93b0005fd0 tx=0x7f93b002a4f0 comp rx=0 tx=0).stop 2024-09-17T03:15:19.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.829+0000 7f93cd0d3640 1 -- 172.21.15.87:0/1777957598 shutdown_connections 2024-09-17T03:15:19.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.829+0000 7f93cd0d3640 1 --2- 172.21.15.87:0/1777957598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93c80b3d60 0x7f93c80b4170 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:19.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.829+0000 7f93cd0d3640 1 --2- 172.21.15.87:0/1777957598 >> v2:172.21.15.149:3300/0 conn(0x7f93c80b7f10 0x7f93c815c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:19.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.829+0000 7f93cd0d3640 1 -- 172.21.15.87:0/1777957598 >> 172.21.15.87:0/1777957598 conn(0x7f93c8069330 msgr2=0x7f93c80ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:19.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.829+0000 7f93cd0d3640 1 -- 172.21.15.87:0/1777957598 shutdown_connections 2024-09-17T03:15:19.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.830+0000 7f93cd0d3640 1 -- 172.21.15.87:0/1777957598 wait complete. 2024-09-17T03:15:19.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.831+0000 7f93cd0d3640 1 Processor -- start 2024-09-17T03:15:19.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.832+0000 7f93cd0d3640 1 -- start start 2024-09-17T03:15:19.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.833+0000 7f93cd0d3640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f93c80710e0 0x7f93c80b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:19.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.833+0000 7f93cd0d3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93c80b3d60 0x7f93c814c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:19.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.833+0000 7f93cd0d3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93c80b7f10 0x7f93c814cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:19.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.833+0000 7f93cd0d3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f93c80b7b10 con 0x7f93c80b3d60 2024-09-17T03:15:19.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.833+0000 7f93cd0d3640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f93c80d1c30 con 0x7f93c80b7f10 2024-09-17T03:15:19.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.833+0000 7f93cd0d3640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f93c80b7c90 con 0x7f93c80710e0 2024-09-17T03:15:19.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.833+0000 7f93be59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93c80b3d60 0x7f93c814c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:19.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.833+0000 7f93be59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93c80b3d60 0x7f93c814c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34872/0 (socket says 172.21.15.87:34872) 2024-09-17T03:15:19.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.833+0000 7f93be59c640 1 -- 172.21.15.87:0/2999304756 learned_addr learned my addr 172.21.15.87:0/2999304756 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:19.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.833+0000 7f93c759e640 1 --2- 172.21.15.87:0/2999304756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93c80b7f10 0x7f93c814cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:19.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.833+0000 7f93c6d9d640 1 --2- 172.21.15.87:0/2999304756 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f93c80710e0 0x7f93c80b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:19.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.833+0000 7f93be59c640 1 -- 172.21.15.87:0/2999304756 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f93c80710e0 msgr2=0x7f93c80b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:19.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.833+0000 7f93be59c640 1 --2- 172.21.15.87:0/2999304756 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f93c80710e0 0x7f93c80b3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:19.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.833+0000 7f93be59c640 1 -- 172.21.15.87:0/2999304756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93c80b7f10 msgr2=0x7f93c814cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:19.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.833+0000 7f93be59c640 1 --2- 172.21.15.87:0/2999304756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93c80b7f10 0x7f93c814cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:19.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.833+0000 7f93be59c640 1 -- 172.21.15.87:0/2999304756 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c80c9df0 con 0x7f93c80b3d60 2024-09-17T03:15:19.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.833+0000 7f93c759e640 1 --2- 172.21.15.87:0/2999304756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93c80b7f10 0x7f93c814cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:19.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.833+0000 7f93c6d9d640 1 --2- 172.21.15.87:0/2999304756 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f93c80710e0 0x7f93c80b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:19.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.833+0000 7f93be59c640 1 --2- 172.21.15.87:0/2999304756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93c80b3d60 0x7f93c814c5f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f93b0000c00 tx=0x7f93b0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:19.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.834+0000 7f93bffff640 1 -- 172.21.15.87:0/2999304756 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b003e030 con 0x7f93c80b3d60 2024-09-17T03:15:19.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.834+0000 7f93bffff640 1 -- 172.21.15.87:0/2999304756 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93b0003e60 con 0x7f93c80b3d60 2024-09-17T03:15:19.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.834+0000 7f93cd0d3640 1 -- 172.21.15.87:0/2999304756 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c80ca0d0 con 0x7f93c80b3d60 2024-09-17T03:15:19.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.834+0000 7f93bffff640 1 -- 172.21.15.87:0/2999304756 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b0035040 con 0x7f93c80b3d60 2024-09-17T03:15:19.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.834+0000 7f93cd0d3640 1 -- 172.21.15.87:0/2999304756 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c80ca690 con 0x7f93c80b3d60 2024-09-17T03:15:19.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.835+0000 7f93cd0d3640 1 -- 172.21.15.87:0/2999304756 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93c814bfa0 con 0x7f93c80b3d60 2024-09-17T03:15:19.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.835+0000 7f93bffff640 1 -- 172.21.15.87:0/2999304756 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f93b0041030 con 0x7f93c80b3d60 2024-09-17T03:15:19.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.836+0000 7f93bffff640 1 --2- 172.21.15.87:0/2999304756 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f93a8078b90 0x7f93a807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:19.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.836+0000 7f93bffff640 1 -- 172.21.15.87:0/2999304756 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7f93b00bb610 con 0x7f93c80b3d60 2024-09-17T03:15:19.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.836+0000 7f93c6d9d640 1 --2- 172.21.15.87:0/2999304756 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f93a8078b90 0x7f93a807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:19.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.837+0000 7f93c6d9d640 1 --2- 172.21.15.87:0/2999304756 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f93a8078b90 0x7f93a807b0a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f93b8005fd0 tx=0x7f93b8005e00 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:19.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:19.841+0000 7f93bffff640 1 -- 172.21.15.87:0/2999304756 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f93c814bfa0 con 0x7f93c80b3d60 2024-09-17T03:15:20.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:20.050+0000 7f93cd0d3640 1 -- 172.21.15.87:0/2999304756 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f93c8071e30 con 0x7f93c80b3d60 2024-09-17T03:15:20.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:20.050+0000 7f93bffff640 1 -- 172.21.15.87:0/2999304756 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) ==== 53+0+82 (secure 0 0 0) 0x7f93c80b46b0 con 0x7f93c80b3d60 2024-09-17T03:15:20.049 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data data_new ] 2024-09-17T03:15:20.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:20.052+0000 7f93cd0d3640 1 -- 172.21.15.87:0/2999304756 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f93a8078b90 msgr2=0x7f93a807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:20.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:20.052+0000 7f93cd0d3640 1 --2- 172.21.15.87:0/2999304756 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f93a8078b90 0x7f93a807b0a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f93b8005fd0 tx=0x7f93b8005e00 comp rx=0 tx=0).stop 2024-09-17T03:15:20.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:20.052+0000 7f93cd0d3640 1 -- 172.21.15.87:0/2999304756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93c80b3d60 msgr2=0x7f93c814c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:20.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:20.052+0000 7f93cd0d3640 1 --2- 172.21.15.87:0/2999304756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93c80b3d60 0x7f93c814c5f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f93b0000c00 tx=0x7f93b0003970 comp rx=0 tx=0).stop 2024-09-17T03:15:20.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:20.052+0000 7f93cd0d3640 1 -- 172.21.15.87:0/2999304756 shutdown_connections 2024-09-17T03:15:20.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:20.052+0000 7f93cd0d3640 1 --2- 172.21.15.87:0/2999304756 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f93a8078b90 0x7f93a807b0a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:20.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:20.052+0000 7f93cd0d3640 1 --2- 172.21.15.87:0/2999304756 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f93c80b7f10 0x7f93c814cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:20.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:20.052+0000 7f93cd0d3640 1 --2- 172.21.15.87:0/2999304756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f93c80b3d60 0x7f93c814c5f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:20.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:20.052+0000 7f93cd0d3640 1 --2- 172.21.15.87:0/2999304756 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f93c80710e0 0x7f93c80b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:20.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:20.052+0000 7f93cd0d3640 1 -- 172.21.15.87:0/2999304756 >> 172.21.15.87:0/2999304756 conn(0x7f93c8069330 msgr2=0x7f93c815ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:20.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:20.052+0000 7f93cd0d3640 1 -- 172.21.15.87:0/2999304756 shutdown_connections 2024-09-17T03:15:20.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:20.052+0000 7f93cd0d3640 1 -- 172.21.15.87:0/2999304756 wait complete. 2024-09-17T03:15:20.062 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T03:15:20.062 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T03:15:20.062 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-09-17T03:15:20.062 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T03:15:20.062 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:20.062 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T03:15:20.063 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T03:15:20.063 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T03:15:20.063 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:15:20.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:20.063 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-09-17T03:15:20.078 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:15:20.078 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:15:20.079 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:15:20.079 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:20.079 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-09-17T03:15:20.079 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:20.079 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:15:20.079 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-09-17T03:15:20.079 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-09-17T03:15:20.079 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:15:20.079 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-09-17T03:15:20.079 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-09-17T03:15:20.079 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:15:20.079 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:15:20.079 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T03:15:20.080 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:15:20.080 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T03:15:20.080 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42712sec preferred_lft 42712sec 2024-09-17T03:15:20.080 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link noprefixroute 2024-09-17T03:15:20.080 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:20.080 INFO:teuthology.orchestra.run.smithi087.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:15:20.080 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 26:32:f7:da:05:b7 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:15:20.080 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T03:15:20.080 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:20.080 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c822:ceff:fef0:4a99/64 scope link 2024-09-17T03:15:20.080 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:20.080 INFO:teuthology.orchestra.run.smithi087.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:15:20.081 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether b2:cb:c1:94:14:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:15:20.081 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::b0cb:c1ff:fe94:145d/64 scope link 2024-09-17T03:15:20.081 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:20.081 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:15:20.081 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 26:32:f7:da:05:b7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:15:20.081 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::2432:f7ff:feda:5b7/64 scope link 2024-09-17T03:15:20.081 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:20.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:20.081 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T03:15:20.141 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-09-17T03:15:20.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:20.143 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-09-17T03:15:20.199 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-09-17T03:15:20.200 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T03:15:20.200 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T03:15:20.200 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T03:15:20.200 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T03:15:20.200 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:15:20.200 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-09-17T03:15:20.200 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T03:15:20.201 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T03:15:20.201 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T03:15:20.201 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-09-17T03:15:20.288 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:15:20.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:15:20.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:20.325 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-09-17T03:15:20.341 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T03:15:20.341 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T03:15:20.342 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:15:20.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:20.342 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-09-17T03:15:20.407 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T03:15:20.407 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:15:20.407 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:20.463 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:15:20.463 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:15:20.464 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:20.518 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-09-17T03:15:20.583 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T03:15:20.641 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-09-17T03:15:20.641 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-09-17T03:15:20.642 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-09-17T03:15:20.642 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-09-17T03:15:20.642 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-09-17T03:15:20.642 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-09-17T03:15:20.642 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-09-17T03:15:20.642 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-09-17T03:15:20.642 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-09-17T03:15:20.642 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-09-17T03:15:20.642 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-09-17T03:15:20.642 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-09-17T03:15:20.642 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T03:15:20.642 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-09-17T03:15:20.642 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-09-17T03:15:20.643 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-09-17T03:15:20.643 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-09-17T03:15:20.643 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-09-17T03:15:20.643 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-09-17T03:15:20.643 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T03:15:20.643 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T03:15:20.643 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T03:15:20.643 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T03:15:20.643 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-09-17T03:15:20.643 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-09-17T03:15:20.643 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T03:15:20.643 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T03:15:20.643 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T03:15:20.644 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T03:15:20.644 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-09-17T03:15:20.644 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:15:20.644 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:15:20.644 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:15:20.644 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:15:20.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:20.644 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-09-17T03:15:20.697 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T03:15:20.697 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-17T03:15:20.739 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-09-17T03:15:20.763 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T03:15:20.821 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-09-17T03:15:20.821 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-09-17T03:15:20.821 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-09-17T03:15:20.821 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-09-17T03:15:20.821 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-09-17T03:15:20.821 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-09-17T03:15:20.821 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-09-17T03:15:20.821 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-09-17T03:15:20.822 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-09-17T03:15:20.822 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-09-17T03:15:20.822 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-09-17T03:15:20.822 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-09-17T03:15:20.822 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T03:15:20.822 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-09-17T03:15:20.822 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-09-17T03:15:20.822 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-09-17T03:15:20.822 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-09-17T03:15:20.822 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-09-17T03:15:20.822 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-09-17T03:15:20.822 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T03:15:20.822 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T03:15:20.822 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T03:15:20.822 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T03:15:20.823 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-09-17T03:15:20.823 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-09-17T03:15:20.823 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T03:15:20.823 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T03:15:20.823 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T03:15:20.823 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T03:15:20.823 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-09-17T03:15:20.823 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:15:20.823 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:15:20.823 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:15:20.823 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:15:20.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:20.824 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-09-17T03:15:20.830 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:2024-09-17T03:15:20.831+0000 7f60a6574480 -1 init, newargv = 0x7f6088004720 newargc=15 2024-09-17T03:15:20.830 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:ceph-fuse[50983]: starting ceph client 2024-09-17T03:15:20.835 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:ceph-fuse[50983]: starting fuse 2024-09-17T03:15:20.845 INFO:teuthology.orchestra.run.smithi087.stdout:42 2024-09-17T03:15:20.845 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-17T03:15:20.846 DEBUG:teuthology.orchestra.run.smithi087:> sudo stdin-killer -- python3 -c ' 2024-09-17T03:15:20.846 DEBUG:teuthology.orchestra.run.smithi087:> import glob 2024-09-17T03:15:20.846 DEBUG:teuthology.orchestra.run.smithi087:> import re 2024-09-17T03:15:20.846 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-09-17T03:15:20.846 DEBUG:teuthology.orchestra.run.smithi087:> import subprocess 2024-09-17T03:15:20.846 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:15:20.846 DEBUG:teuthology.orchestra.run.smithi087:> def _find_admin_socket(client_name): 2024-09-17T03:15:20.846 DEBUG:teuthology.orchestra.run.smithi087:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T03:15:20.846 DEBUG:teuthology.orchestra.run.smithi087:> files = glob.glob(asok_path) 2024-09-17T03:15:20.846 DEBUG:teuthology.orchestra.run.smithi087:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T03:15:20.846 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:15:20.847 DEBUG:teuthology.orchestra.run.smithi087:> # Given a non-glob path, it better be there 2024-09-17T03:15:20.847 DEBUG:teuthology.orchestra.run.smithi087:> if "*" not in asok_path: 2024-09-17T03:15:20.847 DEBUG:teuthology.orchestra.run.smithi087:> assert(len(files) == 1) 2024-09-17T03:15:20.847 DEBUG:teuthology.orchestra.run.smithi087:> return files[0] 2024-09-17T03:15:20.847 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:15:20.847 DEBUG:teuthology.orchestra.run.smithi087:> for f in files: 2024-09-17T03:15:20.847 DEBUG:teuthology.orchestra.run.smithi087:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T03:15:20.847 DEBUG:teuthology.orchestra.run.smithi087:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T03:15:20.847 DEBUG:teuthology.orchestra.run.smithi087:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T03:15:20.847 DEBUG:teuthology.orchestra.run.smithi087:> contents = proc_f.read() 2024-09-17T03:15:20.847 DEBUG:teuthology.orchestra.run.smithi087:> if mountpoint in contents: 2024-09-17T03:15:20.847 DEBUG:teuthology.orchestra.run.smithi087:> return f 2024-09-17T03:15:20.847 DEBUG:teuthology.orchestra.run.smithi087:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T03:15:20.847 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:15:20.847 DEBUG:teuthology.orchestra.run.smithi087:> print(_find_admin_socket("client.0")) 2024-09-17T03:15:20.848 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-09-17T03:15:20.961 INFO:teuthology.orchestra.run.smithi087.stdout:/var/run/ceph/ceph-client.0.50983.asok 2024-09-17T03:15:20.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:15:20.972 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.50983.asok 2024-09-17T03:15:20.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:20.972 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.50983.asok status 2024-09-17T03:15:21.113 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-17T03:15:21.114 INFO:teuthology.orchestra.run.smithi087.stdout: "metadata": { 2024-09-17T03:15:21.114 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T03:15:21.114 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T03:15:21.114 INFO:teuthology.orchestra.run.smithi087.stdout: "entity_id": "0", 2024-09-17T03:15:21.114 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": "smithi087", 2024-09-17T03:15:21.114 INFO:teuthology.orchestra.run.smithi087.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T03:15:21.114 INFO:teuthology.orchestra.run.smithi087.stdout: "pid": "50983", 2024-09-17T03:15:21.114 INFO:teuthology.orchestra.run.smithi087.stdout: "root": "/" 2024-09-17T03:15:21.114 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:21.114 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_count": 0, 2024-09-17T03:15:21.114 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_pinned_count": 0, 2024-09-17T03:15:21.114 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 5313, 2024-09-17T03:15:21.115 INFO:teuthology.orchestra.run.smithi087.stdout: "inst": { 2024-09-17T03:15:21.115 INFO:teuthology.orchestra.run.smithi087.stdout: "name": { 2024-09-17T03:15:21.115 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "client", 2024-09-17T03:15:21.115 INFO:teuthology.orchestra.run.smithi087.stdout: "num": 5313 2024-09-17T03:15:21.115 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:21.115 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-09-17T03:15:21.115 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-09-17T03:15:21.115 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-09-17T03:15:21.115 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 2718043802 2024-09-17T03:15:21.115 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:21.115 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:21.115 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-09-17T03:15:21.115 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-09-17T03:15:21.115 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-09-17T03:15:21.115 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 2718043802 2024-09-17T03:15:21.116 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:21.116 INFO:teuthology.orchestra.run.smithi087.stdout: "inst_str": "client.5313 192.168.0.1:0/2718043802", 2024-09-17T03:15:21.116 INFO:teuthology.orchestra.run.smithi087.stdout: "addr_str": "192.168.0.1:0/2718043802", 2024-09-17T03:15:21.116 INFO:teuthology.orchestra.run.smithi087.stdout: "inode_count": 1, 2024-09-17T03:15:21.116 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_epoch": 14, 2024-09-17T03:15:21.116 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch": 37, 2024-09-17T03:15:21.116 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch_barrier": 0, 2024-09-17T03:15:21.116 INFO:teuthology.orchestra.run.smithi087.stdout: "blocklisted": false, 2024-09-17T03:15:21.116 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs" 2024-09-17T03:15:21.116 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-17T03:15:21.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:21.122 DEBUG:teuthology.orchestra.run.smithi087:> stat --file-system '--printf=%T 2024-09-17T03:15:21.122 DEBUG:teuthology.orchestra.run.smithi087:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:21.138 INFO:teuthology.orchestra.run.smithi087.stdout:fuseblk 2024-09-17T03:15:21.138 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:21.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:21.138 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:21.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:21.204 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.file.layout.pool -v data_new layoutfile) 2024-09-17T03:15:21.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:21.261 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v data_new layoutdir) 2024-09-17T03:15:21.321 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:15:21.321 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:15:21.378 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:15:21.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.531+0000 7fdd5aed3640 1 -- 172.21.15.87:0/2132639083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd54074fb0 msgr2=0x7fdd540c4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:21.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.531+0000 7fdd5aed3640 1 --2- 172.21.15.87:0/2132639083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd54074fb0 0x7fdd540c4f00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c005fd0 tx=0x7fdd4c02ab40 comp rx=0 tx=0).stop 2024-09-17T03:15:21.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.531+0000 7fdd5aed3640 1 -- 172.21.15.87:0/2132639083 shutdown_connections 2024-09-17T03:15:21.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.531+0000 7fdd5aed3640 1 --2- 172.21.15.87:0/2132639083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd54074fb0 0x7fdd540c4f00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:21.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.531+0000 7fdd5aed3640 1 -- 172.21.15.87:0/2132639083 >> 172.21.15.87:0/2132639083 conn(0x7fdd5406ee70 msgr2=0x7fdd5406f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:21.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.531+0000 7fdd5aed3640 1 -- 172.21.15.87:0/2132639083 shutdown_connections 2024-09-17T03:15:21.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.531+0000 7fdd5aed3640 1 -- 172.21.15.87:0/2132639083 wait complete. 2024-09-17T03:15:21.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.533+0000 7fdd5aed3640 1 Processor -- start 2024-09-17T03:15:21.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.533+0000 7fdd5aed3640 1 -- start start 2024-09-17T03:15:21.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.535+0000 7fdd5aed3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd540cc590 0x7fdd540adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:21.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.535+0000 7fdd5aed3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5415f0c0 0x7fdd541f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:21.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.535+0000 7fdd5aed3640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdd541f9250 0x7fdd5415d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:21.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.535+0000 7fdd5aed3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdd540c5440 con 0x7fdd5415f0c0 2024-09-17T03:15:21.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.535+0000 7fdd5aed3640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdd5415f720 con 0x7fdd540cc590 2024-09-17T03:15:21.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.535+0000 7fdd5aed3640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fdd540c55c0 con 0x7fdd541f9250 2024-09-17T03:15:21.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.535+0000 7fdd53fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5415f0c0 0x7fdd541f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:21.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.535+0000 7fdd53fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5415f0c0 0x7fdd541f8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34890/0 (socket says 172.21.15.87:34890) 2024-09-17T03:15:21.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.535+0000 7fdd53fff640 1 -- 172.21.15.87:0/2366216407 learned_addr learned my addr 172.21.15.87:0/2366216407 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:21.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.535+0000 7fdd59470640 1 --2- 172.21.15.87:0/2366216407 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdd541f9250 0x7fdd5415d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:21.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.535+0000 7fdd58c6f640 1 --2- 172.21.15.87:0/2366216407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd540cc590 0x7fdd540adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:21.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.535+0000 7fdd53fff640 1 -- 172.21.15.87:0/2366216407 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdd541f9250 msgr2=0x7fdd5415d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:21.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.535+0000 7fdd53fff640 1 --2- 172.21.15.87:0/2366216407 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdd541f9250 0x7fdd5415d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:21.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.535+0000 7fdd53fff640 1 -- 172.21.15.87:0/2366216407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd540cc590 msgr2=0x7fdd540adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:21.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.535+0000 7fdd53fff640 1 --2- 172.21.15.87:0/2366216407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd540cc590 0x7fdd540adeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:21.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.535+0000 7fdd53fff640 1 -- 172.21.15.87:0/2366216407 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd541f5290 con 0x7fdd5415f0c0 2024-09-17T03:15:21.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.535+0000 7fdd53fff640 1 --2- 172.21.15.87:0/2366216407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5415f0c0 0x7fdd541f8d10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdd4000c370 tx=0x7fdd4000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:21.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.535+0000 7fdd517fa640 1 -- 172.21.15.87:0/2366216407 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd40017070 con 0x7fdd5415f0c0 2024-09-17T03:15:21.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.535+0000 7fdd517fa640 1 -- 172.21.15.87:0/2366216407 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd400035f0 con 0x7fdd5415f0c0 2024-09-17T03:15:21.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.535+0000 7fdd5aed3640 1 -- 172.21.15.87:0/2366216407 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd541f5570 con 0x7fdd5415f0c0 2024-09-17T03:15:21.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.535+0000 7fdd517fa640 1 -- 172.21.15.87:0/2366216407 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd40003a40 con 0x7fdd5415f0c0 2024-09-17T03:15:21.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.536+0000 7fdd5aed3640 1 -- 172.21.15.87:0/2366216407 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd541f5b30 con 0x7fdd5415f0c0 2024-09-17T03:15:21.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.536+0000 7fdd5aed3640 1 -- 172.21.15.87:0/2366216407 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd540ae7c0 con 0x7fdd5415f0c0 2024-09-17T03:15:21.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.537+0000 7fdd517fa640 1 -- 172.21.15.87:0/2366216407 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdd4001a070 con 0x7fdd5415f0c0 2024-09-17T03:15:21.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.538+0000 7fdd517fa640 1 --2- 172.21.15.87:0/2366216407 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdd38078bc0 0x7fdd3807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:21.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.538+0000 7fdd517fa640 1 -- 172.21.15.87:0/2366216407 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7fdd4009a0a0 con 0x7fdd5415f0c0 2024-09-17T03:15:21.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.538+0000 7fdd58c6f640 1 --2- 172.21.15.87:0/2366216407 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdd38078bc0 0x7fdd3807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:21.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.538+0000 7fdd58c6f640 1 --2- 172.21.15.87:0/2366216407 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdd38078bc0 0x7fdd3807b0d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdd48000f60 tx=0x7fdd48009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:21.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.542+0000 7fdd517fa640 1 -- 172.21.15.87:0/2366216407 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fdd40061d10 con 0x7fdd5415f0c0 2024-09-17T03:15:21.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.727+0000 7fdd5aed3640 1 -- 172.21.15.87:0/2366216407 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdd540c5a20 con 0x7fdd5415f0c0 2024-09-17T03:15:21.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.727+0000 7fdd517fa640 1 -- 172.21.15.87:0/2366216407 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) ==== 80+0+0 (secure 0 0 0) 0x7fdd40066740 con 0x7fdd5415f0c0 2024-09-17T03:15:21.726 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-09-17T03:15:21.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.730+0000 7fdd5aed3640 1 -- 172.21.15.87:0/2366216407 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdd38078bc0 msgr2=0x7fdd3807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:21.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.730+0000 7fdd5aed3640 1 --2- 172.21.15.87:0/2366216407 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdd38078bc0 0x7fdd3807b0d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdd48000f60 tx=0x7fdd48009040 comp rx=0 tx=0).stop 2024-09-17T03:15:21.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.730+0000 7fdd5aed3640 1 -- 172.21.15.87:0/2366216407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5415f0c0 msgr2=0x7fdd541f8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:21.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.730+0000 7fdd5aed3640 1 --2- 172.21.15.87:0/2366216407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5415f0c0 0x7fdd541f8d10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdd4000c370 tx=0x7fdd4000c830 comp rx=0 tx=0).stop 2024-09-17T03:15:21.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.730+0000 7fdd5aed3640 1 -- 172.21.15.87:0/2366216407 shutdown_connections 2024-09-17T03:15:21.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.730+0000 7fdd5aed3640 1 --2- 172.21.15.87:0/2366216407 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdd38078bc0 0x7fdd3807b0d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:21.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.730+0000 7fdd5aed3640 1 --2- 172.21.15.87:0/2366216407 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdd541f9250 0x7fdd5415d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:21.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.730+0000 7fdd5aed3640 1 --2- 172.21.15.87:0/2366216407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd5415f0c0 0x7fdd541f8d10 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:21.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.730+0000 7fdd5aed3640 1 --2- 172.21.15.87:0/2366216407 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdd540cc590 0x7fdd540adeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:21.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.730+0000 7fdd5aed3640 1 -- 172.21.15.87:0/2366216407 >> 172.21.15.87:0/2366216407 conn(0x7fdd5406ee70 msgr2=0x7fdd540cf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:21.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.730+0000 7fdd5aed3640 1 -- 172.21.15.87:0/2366216407 shutdown_connections 2024-09-17T03:15:21.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.730+0000 7fdd5aed3640 1 -- 172.21.15.87:0/2366216407 wait complete. 2024-09-17T03:15:21.740 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:15:21.740 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:15:21.756 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:15:21.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.910+0000 7f317cfb0640 1 -- 172.21.15.87:0/2528797839 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31780710e0 msgr2=0x7f31780714f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:21.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.910+0000 7f317cfb0640 1 --2- 172.21.15.87:0/2528797839 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31780710e0 0x7f31780714f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3178072370 tx=0x7f317002a980 comp rx=0 tx=0).stop 2024-09-17T03:15:21.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.910+0000 7f317cfb0640 1 -- 172.21.15.87:0/2528797839 shutdown_connections 2024-09-17T03:15:21.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.910+0000 7f317cfb0640 1 --2- 172.21.15.87:0/2528797839 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31780710e0 0x7f31780714f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:21.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.910+0000 7f317cfb0640 1 --2- 172.21.15.87:0/2528797839 >> v2:172.21.15.149:3300/0 conn(0x7f31780b3d00 0x7f31780b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:21.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.910+0000 7f317cfb0640 1 --2- 172.21.15.87:0/2528797839 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f317815c6c0 0x7f3178160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:21.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.910+0000 7f317cfb0640 1 -- 172.21.15.87:0/2528797839 >> 172.21.15.87:0/2528797839 conn(0x7f3178069360 msgr2=0x7f31780ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:21.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.910+0000 7f317cfb0640 1 -- 172.21.15.87:0/2528797839 shutdown_connections 2024-09-17T03:15:21.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.910+0000 7f317cfb0640 1 -- 172.21.15.87:0/2528797839 wait complete. 2024-09-17T03:15:21.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.912+0000 7f317cfb0640 1 Processor -- start 2024-09-17T03:15:21.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.912+0000 7f317cfb0640 1 -- start start 2024-09-17T03:15:21.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.913+0000 7f317cfb0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31780710e0 0x7f31780b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:21.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.913+0000 7f317cfb0640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31780b3d00 0x7f31780ca380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:21.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.913+0000 7f317cfb0640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f317815c6c0 0x7f31780ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:21.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.913+0000 7f317cfb0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3178071a30 con 0x7f31780710e0 2024-09-17T03:15:21.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.913+0000 7f317cfb0640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f31780b80d0 con 0x7f31780b3d00 2024-09-17T03:15:21.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.913+0000 7f317cfb0640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3178071bb0 con 0x7f317815c6c0 2024-09-17T03:15:21.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.913+0000 7f317659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31780710e0 0x7f31780b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:21.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.913+0000 7f317659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31780710e0 0x7f31780b33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34908/0 (socket says 172.21.15.87:34908) 2024-09-17T03:15:21.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.914+0000 7f317659c640 1 -- 172.21.15.87:0/2845897410 learned_addr learned my addr 172.21.15.87:0/2845897410 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:21.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.914+0000 7f316dd9b640 1 --2- 172.21.15.87:0/2845897410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31780b3d00 0x7f31780ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:21.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.914+0000 7f3176d9d640 1 --2- 172.21.15.87:0/2845897410 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f317815c6c0 0x7f31780ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:21.915 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.914+0000 7f317659c640 1 -- 172.21.15.87:0/2845897410 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f317815c6c0 msgr2=0x7f31780ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:21.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.914+0000 7f317659c640 1 --2- 172.21.15.87:0/2845897410 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f317815c6c0 0x7f31780ca8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:21.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.914+0000 7f317659c640 1 -- 172.21.15.87:0/2845897410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31780b3d00 msgr2=0x7f31780ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:21.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.914+0000 7f317659c640 1 --2- 172.21.15.87:0/2845897410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31780b3d00 0x7f31780ca380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:21.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.914+0000 7f317659c640 1 -- 172.21.15.87:0/2845897410 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317815d160 con 0x7f31780710e0 2024-09-17T03:15:21.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.914+0000 7f317659c640 1 --2- 172.21.15.87:0/2845897410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31780710e0 0x7f31780b33d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f316800d770 tx=0x7f316800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:21.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.914+0000 7f316f7fe640 1 -- 172.21.15.87:0/2845897410 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3168017070 con 0x7f31780710e0 2024-09-17T03:15:21.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.914+0000 7f316f7fe640 1 -- 172.21.15.87:0/2845897410 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3168009d20 con 0x7f31780710e0 2024-09-17T03:15:21.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.914+0000 7f316f7fe640 1 -- 172.21.15.87:0/2845897410 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3168004280 con 0x7f31780710e0 2024-09-17T03:15:21.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.914+0000 7f317cfb0640 1 -- 172.21.15.87:0/2845897410 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f317815ff60 con 0x7f31780710e0 2024-09-17T03:15:21.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.914+0000 7f317cfb0640 1 -- 172.21.15.87:0/2845897410 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f317815d420 con 0x7f31780710e0 2024-09-17T03:15:21.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.915+0000 7f317cfb0640 1 -- 172.21.15.87:0/2845897410 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31780b4110 con 0x7f31780710e0 2024-09-17T03:15:21.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.916+0000 7f316f7fe640 1 -- 172.21.15.87:0/2845897410 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3168005020 con 0x7f31780710e0 2024-09-17T03:15:21.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.916+0000 7f316f7fe640 1 --2- 172.21.15.87:0/2845897410 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3158078bc0 0x7f315807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:21.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.917+0000 7f316f7fe640 1 -- 172.21.15.87:0/2845897410 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7f316809b150 con 0x7f31780710e0 2024-09-17T03:15:21.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.917+0000 7f316dd9b640 1 --2- 172.21.15.87:0/2845897410 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3158078bc0 0x7f315807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:21.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.917+0000 7f316dd9b640 1 --2- 172.21.15.87:0/2845897410 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3158078bc0 0x7f315807b0d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f317815d0f0 tx=0x7f3160005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:21.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:21.921+0000 7f316f7fe640 1 -- 172.21.15.87:0/2845897410 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3168020020 con 0x7f31780710e0 2024-09-17T03:15:22.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.108+0000 7f317cfb0640 1 -- 172.21.15.87:0/2845897410 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f317815bec0 con 0x7f31780710e0 2024-09-17T03:15:22.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.108+0000 7f316f7fe640 1 -- 172.21.15.87:0/2845897410 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) ==== 80+0+0 (secure 0 0 0) 0x7f3168062e40 con 0x7f31780710e0 2024-09-17T03:15:22.107 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-09-17T03:15:22.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.110+0000 7f317cfb0640 1 -- 172.21.15.87:0/2845897410 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3158078bc0 msgr2=0x7f315807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:22.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.110+0000 7f317cfb0640 1 --2- 172.21.15.87:0/2845897410 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3158078bc0 0x7f315807b0d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f317815d0f0 tx=0x7f3160005f90 comp rx=0 tx=0).stop 2024-09-17T03:15:22.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.110+0000 7f317cfb0640 1 -- 172.21.15.87:0/2845897410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31780710e0 msgr2=0x7f31780b33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:22.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.110+0000 7f317cfb0640 1 --2- 172.21.15.87:0/2845897410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31780710e0 0x7f31780b33d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f316800d770 tx=0x7f316800dc30 comp rx=0 tx=0).stop 2024-09-17T03:15:22.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.110+0000 7f317cfb0640 1 -- 172.21.15.87:0/2845897410 shutdown_connections 2024-09-17T03:15:22.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.110+0000 7f317cfb0640 1 --2- 172.21.15.87:0/2845897410 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3158078bc0 0x7f315807b0d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:22.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.110+0000 7f317cfb0640 1 --2- 172.21.15.87:0/2845897410 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f317815c6c0 0x7f31780ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:22.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.110+0000 7f317cfb0640 1 --2- 172.21.15.87:0/2845897410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f31780b3d00 0x7f31780ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:22.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.110+0000 7f317cfb0640 1 --2- 172.21.15.87:0/2845897410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31780710e0 0x7f31780b33d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:22.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.110+0000 7f317cfb0640 1 -- 172.21.15.87:0/2845897410 >> 172.21.15.87:0/2845897410 conn(0x7f3178069360 msgr2=0x7f317806e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:22.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.110+0000 7f317cfb0640 1 -- 172.21.15.87:0/2845897410 shutdown_connections 2024-09-17T03:15:22.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.110+0000 7f317cfb0640 1 -- 172.21.15.87:0/2845897410 wait complete. 2024-09-17T03:15:22.120 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi087.front.sepia.ceph.com... 2024-09-17T03:15:22.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:22.120 DEBUG:teuthology.orchestra.run.smithi087:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:22.143 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:ceph-fuse[50983]: fuse finished with error 0 and tester_r 0 2024-09-17T03:15:22.145 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:15:22.209 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-09-17T03:15:22.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:22.210 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T03:15:22.225 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2024-09-17T03:15:22.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.386+0000 7fe79f5c3640 1 -- 172.21.15.87:0/946768290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe798072da0 msgr2=0x7fe79815ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:22.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.386+0000 7fe79f5c3640 1 --2- 172.21.15.87:0/946768290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe798072da0 0x7fe79815ef40 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe794005fd0 tx=0x7fe79402a5d0 comp rx=0 tx=0).stop 2024-09-17T03:15:22.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.386+0000 7fe79f5c3640 1 -- 172.21.15.87:0/946768290 shutdown_connections 2024-09-17T03:15:22.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.386+0000 7fe79f5c3640 1 --2- 172.21.15.87:0/946768290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe798072da0 0x7fe79815ef40 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:22.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.386+0000 7fe79f5c3640 1 --2- 172.21.15.87:0/946768290 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe79815fb50 0x7fe798152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:22.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.386+0000 7fe79f5c3640 1 -- 172.21.15.87:0/946768290 >> 172.21.15.87:0/946768290 conn(0x7fe7980b1c40 msgr2=0x7fe7980b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:22.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.386+0000 7fe79f5c3640 1 -- 172.21.15.87:0/946768290 shutdown_connections 2024-09-17T03:15:22.385 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.386+0000 7fe79f5c3640 1 -- 172.21.15.87:0/946768290 wait complete. 2024-09-17T03:15:22.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.388+0000 7fe79f5c3640 1 Processor -- start 2024-09-17T03:15:22.386 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.388+0000 7fe79f5c3640 1 -- start start 2024-09-17T03:15:22.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.389+0000 7fe79f5c3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe798072da0 0x7fe798148330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:22.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.389+0000 7fe79f5c3640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe7980731b0 0x7fe798150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:22.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.389+0000 7fe79f5c3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe79815fb50 0x7fe7981513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:22.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.389+0000 7fe79f5c3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe79815c060 con 0x7fe79815fb50 2024-09-17T03:15:22.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.389+0000 7fe79f5c3640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe79815f880 con 0x7fe798072da0 2024-09-17T03:15:22.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.389+0000 7fe79f5c3640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe79815c1e0 con 0x7fe7980731b0 2024-09-17T03:15:22.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.389+0000 7fe79db60640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe79815fb50 0x7fe7981513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:22.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.389+0000 7fe79db60640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe79815fb50 0x7fe7981513e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34932/0 (socket says 172.21.15.87:34932) 2024-09-17T03:15:22.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.389+0000 7fe79db60640 1 -- 172.21.15.87:0/1117235679 learned_addr learned my addr 172.21.15.87:0/1117235679 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:22.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.389+0000 7fe79d35f640 1 --2- 172.21.15.87:0/1117235679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe798072da0 0x7fe798148330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:22.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.389+0000 7fe79cb5e640 1 --2- 172.21.15.87:0/1117235679 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe7980731b0 0x7fe798150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:22.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.389+0000 7fe79db60640 1 -- 172.21.15.87:0/1117235679 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe7980731b0 msgr2=0x7fe798150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:22.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.389+0000 7fe79db60640 1 --2- 172.21.15.87:0/1117235679 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe7980731b0 0x7fe798150ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:22.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.389+0000 7fe79db60640 1 -- 172.21.15.87:0/1117235679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe798072da0 msgr2=0x7fe798148330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:22.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.389+0000 7fe79db60640 1 --2- 172.21.15.87:0/1117235679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe798072da0 0x7fe798148330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:22.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.389+0000 7fe79db60640 1 -- 172.21.15.87:0/1117235679 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79814b7a0 con 0x7fe79815fb50 2024-09-17T03:15:22.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.389+0000 7fe79db60640 1 --2- 172.21.15.87:0/1117235679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe79815fb50 0x7fe7981513e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe79000c760 tx=0x7fe79000cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:22.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.390+0000 7fe789ffb640 1 -- 172.21.15.87:0/1117235679 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe790003b70 con 0x7fe79815fb50 2024-09-17T03:15:22.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.390+0000 7fe789ffb640 1 -- 172.21.15.87:0/1117235679 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe790003d10 con 0x7fe79815fb50 2024-09-17T03:15:22.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.390+0000 7fe79f5c3640 1 -- 172.21.15.87:0/1117235679 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe798151a20 con 0x7fe79815fb50 2024-09-17T03:15:22.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.390+0000 7fe789ffb640 1 -- 172.21.15.87:0/1117235679 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7900052d0 con 0x7fe79815fb50 2024-09-17T03:15:22.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.390+0000 7fe79f5c3640 1 -- 172.21.15.87:0/1117235679 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe79814afa0 con 0x7fe79815fb50 2024-09-17T03:15:22.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.391+0000 7fe79f5c3640 1 -- 172.21.15.87:0/1117235679 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe79815c6d0 con 0x7fe79815fb50 2024-09-17T03:15:22.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.391+0000 7fe789ffb640 1 -- 172.21.15.87:0/1117235679 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe790005470 con 0x7fe79815fb50 2024-09-17T03:15:22.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.392+0000 7fe789ffb640 1 --2- 172.21.15.87:0/1117235679 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe77c078b90 0x7fe77c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:22.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.392+0000 7fe789ffb640 1 -- 172.21.15.87:0/1117235679 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7fe79009ac60 con 0x7fe79815fb50 2024-09-17T03:15:22.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.392+0000 7fe79d35f640 1 --2- 172.21.15.87:0/1117235679 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe77c078b90 0x7fe77c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:22.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.393+0000 7fe79d35f640 1 --2- 172.21.15.87:0/1117235679 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe77c078b90 0x7fe77c07b0a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00b810 tx=0x7fe78c008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:22.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.396+0000 7fe789ffb640 1 -- 172.21.15.87:0/1117235679 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe7980adfd0 con 0x7fe79815fb50 2024-09-17T03:15:22.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.639+0000 7fe79f5c3640 1 -- 172.21.15.87:0/1117235679 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) -- 0x7fe7980b5080 con 0x7fe79815fb50 2024-09-17T03:15:22.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.642+0000 7fe789ffb640 1 -- 172.21.15.87:0/1117235679 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v16) ==== 184+0+141 (secure 0 0 0) 0x7fe790062f10 con 0x7fe79815fb50 2024-09-17T03:15:22.641 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-09-17T03:15:22.641 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDH8+hmLskNLhAAd2M5ynGP8W6Pim11F18wYA== 2024-09-17T03:15:22.641 INFO:teuthology.orchestra.run.smithi087.stdout: caps mds = "allow" 2024-09-17T03:15:22.641 INFO:teuthology.orchestra.run.smithi087.stdout: caps mon = "allow r" 2024-09-17T03:15:22.641 INFO:teuthology.orchestra.run.smithi087.stdout: caps osd = "allow rw pool=cephfs_data" 2024-09-17T03:15:22.641 INFO:teuthology.orchestra.run.smithi087.stderr:updated caps for client.0 2024-09-17T03:15:22.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.644+0000 7fe79f5c3640 1 -- 172.21.15.87:0/1117235679 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe77c078b90 msgr2=0x7fe77c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:22.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.644+0000 7fe79f5c3640 1 --2- 172.21.15.87:0/1117235679 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe77c078b90 0x7fe77c07b0a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00b810 tx=0x7fe78c008040 comp rx=0 tx=0).stop 2024-09-17T03:15:22.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.644+0000 7fe79f5c3640 1 -- 172.21.15.87:0/1117235679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe79815fb50 msgr2=0x7fe7981513e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:22.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.644+0000 7fe79f5c3640 1 --2- 172.21.15.87:0/1117235679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe79815fb50 0x7fe7981513e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe79000c760 tx=0x7fe79000cb60 comp rx=0 tx=0).stop 2024-09-17T03:15:22.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.644+0000 7fe79f5c3640 1 -- 172.21.15.87:0/1117235679 shutdown_connections 2024-09-17T03:15:22.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.644+0000 7fe79f5c3640 1 --2- 172.21.15.87:0/1117235679 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe77c078b90 0x7fe77c07b0a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:22.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.644+0000 7fe79f5c3640 1 --2- 172.21.15.87:0/1117235679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe79815fb50 0x7fe7981513e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:22.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.644+0000 7fe79f5c3640 1 --2- 172.21.15.87:0/1117235679 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe7980731b0 0x7fe798150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:22.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.644+0000 7fe79f5c3640 1 --2- 172.21.15.87:0/1117235679 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe798072da0 0x7fe798148330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:22.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.644+0000 7fe79f5c3640 1 -- 172.21.15.87:0/1117235679 >> 172.21.15.87:0/1117235679 conn(0x7fe7980b1c40 msgr2=0x7fe79815a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:22.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.644+0000 7fe79f5c3640 1 -- 172.21.15.87:0/1117235679 shutdown_connections 2024-09-17T03:15:22.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.644+0000 7fe79f5c3640 1 -- 172.21.15.87:0/1117235679 wait complete. 2024-09-17T03:15:22.654 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T03:15:22.655 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:15:22.697 DEBUG:teuthology.orchestra.run.smithi149:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:15:22.754 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T03:15:22.754 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:15:22.755 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:15:22.755 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:15:22.771 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=156 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:15:22.771 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:15:22.771 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:15:22.771 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:15:22.789 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=1994716,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=3199368k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20815 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=1599680k,nr_inodes=399920,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=3199368k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:15:22.789 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:15:22.789 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:15:22.789 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:15:22.827 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=156 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:15:22.828 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:15:22.828 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:15:22.828 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:15:22.846 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=1994716,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=3199368k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20815 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=1599680k,nr_inodes=399920,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=3199368k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:15:22.846 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:15:22.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.981+0000 7f073faf9640 1 -- 172.21.15.87:0/963220675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07380b6600 msgr2=0x7f073815f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:22.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.981+0000 7f073faf9640 1 --2- 172.21.15.87:0/963220675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07380b6600 0x7f073815f100 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f073808d3e0 tx=0x7f073002a8b0 comp rx=0 tx=0).stop 2024-09-17T03:15:22.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.981+0000 7f073faf9640 1 -- 172.21.15.87:0/963220675 shutdown_connections 2024-09-17T03:15:22.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.981+0000 7f073faf9640 1 --2- 172.21.15.87:0/963220675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07380b6600 0x7f073815f100 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:22.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.981+0000 7f073faf9640 1 --2- 172.21.15.87:0/963220675 >> v2:172.21.15.149:3300/0 conn(0x7f073815c2d0 0x7f07380cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:22.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.981+0000 7f073faf9640 1 -- 172.21.15.87:0/963220675 >> 172.21.15.87:0/963220675 conn(0x7f07380d4470 msgr2=0x7f07380d4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:22.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.981+0000 7f073faf9640 1 -- 172.21.15.87:0/963220675 shutdown_connections 2024-09-17T03:15:22.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.981+0000 7f073faf9640 1 -- 172.21.15.87:0/963220675 wait complete. 2024-09-17T03:15:22.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.983+0000 7f073faf9640 1 Processor -- start 2024-09-17T03:15:22.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.983+0000 7f073faf9640 1 -- start start 2024-09-17T03:15:22.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.984+0000 7f073faf9640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f073815c2d0 0x7f07380cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:22.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.984+0000 7f073faf9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f073815ecf0 0x7f07381611a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:22.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.984+0000 7f073faf9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07381616e0 0x7f0738158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:22.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.984+0000 7f073faf9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f07380d2db0 con 0x7f07381616e0 2024-09-17T03:15:22.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.984+0000 7f073faf9640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f07380d2c30 con 0x7f073815ecf0 2024-09-17T03:15:22.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.984+0000 7f073faf9640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f07380d2f30 con 0x7f073815c2d0 2024-09-17T03:15:22.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.984+0000 7f073e096640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07381616e0 0x7f0738158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:22.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.984+0000 7f073e096640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07381616e0 0x7f0738158440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34956/0 (socket says 172.21.15.87:34956) 2024-09-17T03:15:22.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.984+0000 7f073e096640 1 -- 172.21.15.87:0/454142156 learned_addr learned my addr 172.21.15.87:0/454142156 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:22.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.984+0000 7f073d094640 1 --2- 172.21.15.87:0/454142156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f073815ecf0 0x7f07381611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:22.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.984+0000 7f073d895640 1 --2- 172.21.15.87:0/454142156 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f073815c2d0 0x7f07380cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:22.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.984+0000 7f073e096640 1 -- 172.21.15.87:0/454142156 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f073815c2d0 msgr2=0x7f07380cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:22.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.984+0000 7f073e096640 1 --2- 172.21.15.87:0/454142156 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f073815c2d0 0x7f07380cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:22.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.985+0000 7f073e096640 1 -- 172.21.15.87:0/454142156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f073815ecf0 msgr2=0x7f07381611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:22.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.985+0000 7f073e096640 1 --2- 172.21.15.87:0/454142156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f073815ecf0 0x7f07381611a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:22.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.985+0000 7f073e096640 1 -- 172.21.15.87:0/454142156 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07380cb280 con 0x7f07381616e0 2024-09-17T03:15:22.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.985+0000 7f073d094640 1 --2- 172.21.15.87:0/454142156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f073815ecf0 0x7f07381611a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:15:22.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.985+0000 7f073d895640 1 --2- 172.21.15.87:0/454142156 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f073815c2d0 0x7f07380cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:15:22.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.985+0000 7f073e096640 1 --2- 172.21.15.87:0/454142156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07381616e0 0x7f0738158440 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f073002aea0 tx=0x7f0730003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:22.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.985+0000 7f072e7fc640 1 -- 172.21.15.87:0/454142156 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730031b90 con 0x7f07381616e0 2024-09-17T03:15:22.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.985+0000 7f072e7fc640 1 -- 172.21.15.87:0/454142156 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0730031e20 con 0x7f07381616e0 2024-09-17T03:15:22.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.985+0000 7f073faf9640 1 -- 172.21.15.87:0/454142156 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07381f8950 con 0x7f07381616e0 2024-09-17T03:15:22.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.985+0000 7f072e7fc640 1 -- 172.21.15.87:0/454142156 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07300332f0 con 0x7f07381616e0 2024-09-17T03:15:22.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.985+0000 7f073faf9640 1 -- 172.21.15.87:0/454142156 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07381f8f10 con 0x7f07381616e0 2024-09-17T03:15:22.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.986+0000 7f073faf9640 1 -- 172.21.15.87:0/454142156 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07381f86c0 con 0x7f07381616e0 2024-09-17T03:15:22.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.987+0000 7f072e7fc640 1 -- 172.21.15.87:0/454142156 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f07300316a0 con 0x7f07381616e0 2024-09-17T03:15:22.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.987+0000 7f072e7fc640 1 --2- 172.21.15.87:0/454142156 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f071c078bc0 0x7f071c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:22.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.987+0000 7f072e7fc640 1 -- 172.21.15.87:0/454142156 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7f07300bc500 con 0x7f07381616e0 2024-09-17T03:15:22.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.987+0000 7f073d895640 1 --2- 172.21.15.87:0/454142156 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f071c078bc0 0x7f071c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:22.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.988+0000 7f073d895640 1 --2- 172.21.15.87:0/454142156 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f071c078bc0 0x7f071c07b0d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f07340058c0 tx=0x7f0734005c90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:22.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:22.992+0000 7f072e7fc640 1 -- 172.21.15.87:0/454142156 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f073003f030 con 0x7f07381616e0 2024-09-17T03:15:23.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.202+0000 7f073faf9640 1 -- 172.21.15.87:0/454142156 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07380d3450 con 0x7f07381616e0 2024-09-17T03:15:23.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.202+0000 7f072e7fc640 1 -- 172.21.15.87:0/454142156 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5228 (secure 0 0 0) 0x7f07300847b0 con 0x7f07381616e0 2024-09-17T03:15:23.201 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:23.201 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":14,"btime":"2024-09-17T03:15:08:564652+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6836/1081794135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1081794135},{"type":"v1","addr":"172.21.15.149:6836","nonce":1081794135}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2152665509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6835","nonce":2152665509},{"type":"v1","addr":"172.21.15.149:6837","nonce":2152665509}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/704821751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":704821751},{"type":"v1","addr":"172.21.15.87:6835","nonce":704821751}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:14:55.675616+0000","modified":"2024-09-17T03:15:07.917703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5004},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5004":{"gid":5004,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6834/1243349323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":1243349323},{"type":"v1","addr":"172.21.15.87:6834","nonce":1243349323}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5004,"qdb_cluster":[5004]},"id":2}]} 2024-09-17T03:15:23.202 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 14 2024-09-17T03:15:23.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.204+0000 7f073faf9640 1 -- 172.21.15.87:0/454142156 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f071c078bc0 msgr2=0x7f071c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:23.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.204+0000 7f073faf9640 1 --2- 172.21.15.87:0/454142156 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f071c078bc0 0x7f071c07b0d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f07340058c0 tx=0x7f0734005c90 comp rx=0 tx=0).stop 2024-09-17T03:15:23.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.204+0000 7f073faf9640 1 -- 172.21.15.87:0/454142156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07381616e0 msgr2=0x7f0738158440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:23.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.204+0000 7f073faf9640 1 --2- 172.21.15.87:0/454142156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07381616e0 0x7f0738158440 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f073002aea0 tx=0x7f0730003ba0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.204+0000 7f073faf9640 1 -- 172.21.15.87:0/454142156 shutdown_connections 2024-09-17T03:15:23.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.204+0000 7f073faf9640 1 --2- 172.21.15.87:0/454142156 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f071c078bc0 0x7f071c07b0d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.204+0000 7f073faf9640 1 --2- 172.21.15.87:0/454142156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07381616e0 0x7f0738158440 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.205+0000 7f073faf9640 1 --2- 172.21.15.87:0/454142156 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f073815ecf0 0x7f07381611a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.205+0000 7f073faf9640 1 --2- 172.21.15.87:0/454142156 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f073815c2d0 0x7f07380cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.205+0000 7f073faf9640 1 -- 172.21.15.87:0/454142156 >> 172.21.15.87:0/454142156 conn(0x7f07380d4470 msgr2=0x7f07380d4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:23.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.205+0000 7f073faf9640 1 -- 172.21.15.87:0/454142156 shutdown_connections 2024-09-17T03:15:23.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.205+0000 7f073faf9640 1 -- 172.21.15.87:0/454142156 wait complete. 2024-09-17T03:15:23.215 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:15:23.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.329+0000 7fc8e2313640 1 -- 172.21.15.87:0/2087119312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc0d3060 msgr2=0x7fc8dc0cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:23.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.329+0000 7fc8e2313640 1 --2- 172.21.15.87:0/2087119312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc0d3060 0x7fc8dc0cf6c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4005fd0 tx=0x7fc8d402a820 comp rx=0 tx=0).stop 2024-09-17T03:15:23.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.329+0000 7fc8e2313640 1 -- 172.21.15.87:0/2087119312 shutdown_connections 2024-09-17T03:15:23.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.329+0000 7fc8e2313640 1 --2- 172.21.15.87:0/2087119312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc0d3060 0x7fc8dc0cf6c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.329+0000 7fc8e2313640 1 -- 172.21.15.87:0/2087119312 >> 172.21.15.87:0/2087119312 conn(0x7fc8dc08dae0 msgr2=0x7fc8dc09d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:23.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.329+0000 7fc8e2313640 1 -- 172.21.15.87:0/2087119312 shutdown_connections 2024-09-17T03:15:23.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.329+0000 7fc8e2313640 1 -- 172.21.15.87:0/2087119312 wait complete. 2024-09-17T03:15:23.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.331+0000 7fc8e2313640 1 Processor -- start 2024-09-17T03:15:23.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.331+0000 7fc8e2313640 1 -- start start 2024-09-17T03:15:23.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.332+0000 7fc8e2313640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc8dc0d2c50 0x7fc8dc09e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:23.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.332+0000 7fc8e2313640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8dc0d3060 0x7fc8dc14c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:23.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.332+0000 7fc8e2313640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc0d4860 0x7fc8dc14cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:23.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.332+0000 7fc8e2313640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc0cfd80 con 0x7fc8dc0d4860 2024-09-17T03:15:23.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.332+0000 7fc8e2313640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc0cfc00 con 0x7fc8dc0d3060 2024-09-17T03:15:23.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.332+0000 7fc8e2313640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc8dc0cff00 con 0x7fc8dc0d2c50 2024-09-17T03:15:23.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.332+0000 7fc8e08b0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc0d4860 0x7fc8dc14cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:23.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.332+0000 7fc8e08b0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc0d4860 0x7fc8dc14cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34986/0 (socket says 172.21.15.87:34986) 2024-09-17T03:15:23.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.332+0000 7fc8e08b0640 1 -- 172.21.15.87:0/1486113307 learned_addr learned my addr 172.21.15.87:0/1486113307 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:23.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.333+0000 7fc8e08b0640 1 -- 172.21.15.87:0/1486113307 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc8dc0d2c50 msgr2=0x7fc8dc09e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:23.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.333+0000 7fc8e08b0640 1 --2- 172.21.15.87:0/1486113307 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc8dc0d2c50 0x7fc8dc09e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.333+0000 7fc8e08b0640 1 -- 172.21.15.87:0/1486113307 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8dc0d3060 msgr2=0x7fc8dc14c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:23.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.333+0000 7fc8e08b0640 1 --2- 172.21.15.87:0/1486113307 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8dc0d3060 0x7fc8dc14c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.333+0000 7fc8e08b0640 1 -- 172.21.15.87:0/1486113307 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8dc09eb90 con 0x7fc8dc0d4860 2024-09-17T03:15:23.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.333+0000 7fc8e08b0640 1 --2- 172.21.15.87:0/1486113307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc0d4860 0x7fc8dc14cd40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009b30 tx=0x7fc8d4032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:23.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.333+0000 7fc8d8ff9640 1 -- 172.21.15.87:0/1486113307 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4037040 con 0x7fc8dc0d4860 2024-09-17T03:15:23.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.333+0000 7fc8d8ff9640 1 -- 172.21.15.87:0/1486113307 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8d4042070 con 0x7fc8dc0d4860 2024-09-17T03:15:23.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.333+0000 7fc8e2313640 1 -- 172.21.15.87:0/1486113307 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8dc15f340 con 0x7fc8dc0d4860 2024-09-17T03:15:23.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.333+0000 7fc8d8ff9640 1 -- 172.21.15.87:0/1486113307 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d40042a0 con 0x7fc8dc0d4860 2024-09-17T03:15:23.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.333+0000 7fc8e2313640 1 -- 172.21.15.87:0/1486113307 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8dc15f900 con 0x7fc8dc0d4860 2024-09-17T03:15:23.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.334+0000 7fc8e2313640 1 -- 172.21.15.87:0/1486113307 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8dc0d4150 con 0x7fc8dc0d4860 2024-09-17T03:15:23.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.335+0000 7fc8d8ff9640 1 -- 172.21.15.87:0/1486113307 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc8d4004440 con 0x7fc8dc0d4860 2024-09-17T03:15:23.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.335+0000 7fc8d8ff9640 1 --2- 172.21.15.87:0/1486113307 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc8c8078b90 0x7fc8c807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:23.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.335+0000 7fc8d8ff9640 1 -- 172.21.15.87:0/1486113307 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7fc8d40bbd10 con 0x7fc8dc0d4860 2024-09-17T03:15:23.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.335+0000 7fc8dbfff640 1 --2- 172.21.15.87:0/1486113307 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc8c8078b90 0x7fc8c807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:23.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.336+0000 7fc8dbfff640 1 --2- 172.21.15.87:0/1486113307 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc8c8078b90 0x7fc8c807b0a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc009650 tx=0x7fc8cc006ba0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:23.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.340+0000 7fc8d8ff9640 1 -- 172.21.15.87:0/1486113307 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc8d4083fd0 con 0x7fc8dc0d4860 2024-09-17T03:15:23.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.546+0000 7fc8e2313640 1 -- 172.21.15.87:0/1486113307 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8dc15fad0 con 0x7fc8dc0d4860 2024-09-17T03:15:23.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.547+0000 7fc8d8ff9640 1 -- 172.21.15.87:0/1486113307 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5228 (secure 0 0 0) 0x7fc8d40838d0 con 0x7fc8dc0d4860 2024-09-17T03:15:23.546 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:23.546 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":14,"btime":"2024-09-17T03:15:08:564652+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6836/1081794135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1081794135},{"type":"v1","addr":"172.21.15.149:6836","nonce":1081794135}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2152665509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6835","nonce":2152665509},{"type":"v1","addr":"172.21.15.149:6837","nonce":2152665509}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/704821751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":704821751},{"type":"v1","addr":"172.21.15.87:6835","nonce":704821751}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:14:55.675616+0000","modified":"2024-09-17T03:15:07.917703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5004},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5004":{"gid":5004,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6834/1243349323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":1243349323},{"type":"v1","addr":"172.21.15.87:6834","nonce":1243349323}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5004,"qdb_cluster":[5004]},"id":2}]} 2024-09-17T03:15:23.546 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 14 2024-09-17T03:15:23.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.549+0000 7fc8e2313640 1 -- 172.21.15.87:0/1486113307 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc8c8078b90 msgr2=0x7fc8c807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:23.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.549+0000 7fc8e2313640 1 --2- 172.21.15.87:0/1486113307 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc8c8078b90 0x7fc8c807b0a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc009650 tx=0x7fc8cc006ba0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.549+0000 7fc8e2313640 1 -- 172.21.15.87:0/1486113307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc0d4860 msgr2=0x7fc8dc14cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:23.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.549+0000 7fc8e2313640 1 --2- 172.21.15.87:0/1486113307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc0d4860 0x7fc8dc14cd40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009b30 tx=0x7fc8d4032c90 comp rx=0 tx=0).stop 2024-09-17T03:15:23.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.549+0000 7fc8e2313640 1 -- 172.21.15.87:0/1486113307 shutdown_connections 2024-09-17T03:15:23.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.549+0000 7fc8e2313640 1 --2- 172.21.15.87:0/1486113307 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc8c8078b90 0x7fc8c807b0a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.549+0000 7fc8e2313640 1 --2- 172.21.15.87:0/1486113307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc0d4860 0x7fc8dc14cd40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.549+0000 7fc8e2313640 1 --2- 172.21.15.87:0/1486113307 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8dc0d3060 0x7fc8dc14c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.549+0000 7fc8e2313640 1 --2- 172.21.15.87:0/1486113307 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc8dc0d2c50 0x7fc8dc09e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.549+0000 7fc8e2313640 1 -- 172.21.15.87:0/1486113307 >> 172.21.15.87:0/1486113307 conn(0x7fc8dc08dae0 msgr2=0x7fc8dc0b8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:23.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.549+0000 7fc8e2313640 1 -- 172.21.15.87:0/1486113307 shutdown_connections 2024-09-17T03:15:23.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.549+0000 7fc8e2313640 1 -- 172.21.15.87:0/1486113307 wait complete. 2024-09-17T03:15:23.559 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:15:23.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.674+0000 7f5285d58640 1 -- 172.21.15.87:0/1893491329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52800b3d00 msgr2=0x7f52800b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:23.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.674+0000 7f5285d58640 1 --2- 172.21.15.87:0/1893491329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52800b3d00 0x7f52800b4110 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5274005fd0 tx=0x7f527402a540 comp rx=0 tx=0).stop 2024-09-17T03:15:23.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.674+0000 7f5285d58640 1 -- 172.21.15.87:0/1893491329 shutdown_connections 2024-09-17T03:15:23.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.674+0000 7f5285d58640 1 --2- 172.21.15.87:0/1893491329 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52800d0460 0x7f52800d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.674+0000 7f5285d58640 1 --2- 172.21.15.87:0/1893491329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52800b3d00 0x7f52800b4110 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.674+0000 7f5285d58640 1 -- 172.21.15.87:0/1893491329 >> 172.21.15.87:0/1893491329 conn(0x7f5280069360 msgr2=0x7f52800ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:23.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.674+0000 7f5285d58640 1 -- 172.21.15.87:0/1893491329 shutdown_connections 2024-09-17T03:15:23.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.674+0000 7f5285d58640 1 -- 172.21.15.87:0/1893491329 wait complete. 2024-09-17T03:15:23.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.675+0000 7f5285d58640 1 Processor -- start 2024-09-17T03:15:23.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.676+0000 7f5285d58640 1 -- start start 2024-09-17T03:15:23.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.677+0000 7f5285d58640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52800b3d00 0x7f528014e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:23.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.677+0000 7f5285d58640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52800d0460 0x7f52801f35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:23.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.677+0000 7f5285d58640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f528015c6c0 0x7f52800c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:23.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.677+0000 7f5285d58640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f52800d0f30 con 0x7f52800b3d00 2024-09-17T03:15:23.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.677+0000 7f5285d58640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f52800d0db0 con 0x7f528015c6c0 2024-09-17T03:15:23.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.677+0000 7f5285d58640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f52800d10b0 con 0x7f52800d0460 2024-09-17T03:15:23.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.677+0000 7f527f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52800b3d00 0x7f528014e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:23.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.677+0000 7f527f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52800b3d00 0x7f528014e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34998/0 (socket says 172.21.15.87:34998) 2024-09-17T03:15:23.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.677+0000 7f527f7fe640 1 -- 172.21.15.87:0/750958590 learned_addr learned my addr 172.21.15.87:0/750958590 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:23.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.677+0000 7f527f7fe640 1 -- 172.21.15.87:0/750958590 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52800d0460 msgr2=0x7f52801f35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:23.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.677+0000 7f527ffff640 1 --2- 172.21.15.87:0/750958590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f528015c6c0 0x7f52800c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:23.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.677+0000 7f527f7fe640 1 --2- 172.21.15.87:0/750958590 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52800d0460 0x7f52801f35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.677+0000 7f527f7fe640 1 -- 172.21.15.87:0/750958590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f528015c6c0 msgr2=0x7f52800c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:23.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.677+0000 7f527f7fe640 1 --2- 172.21.15.87:0/750958590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f528015c6c0 0x7f52800c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.677+0000 7f527f7fe640 1 -- 172.21.15.87:0/750958590 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52801f40d0 con 0x7f52800b3d00 2024-09-17T03:15:23.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.677+0000 7f527f7fe640 1 --2- 172.21.15.87:0/750958590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52800b3d00 0x7f528014e870 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f527000c3a0 tx=0x7f527000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:23.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.678+0000 7f5267fff640 1 -- 172.21.15.87:0/750958590 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5270017070 con 0x7f52800b3d00 2024-09-17T03:15:23.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.678+0000 7f5267fff640 1 -- 172.21.15.87:0/750958590 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52700053c0 con 0x7f52800b3d00 2024-09-17T03:15:23.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.678+0000 7f5285d58640 1 -- 172.21.15.87:0/750958590 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52801f43b0 con 0x7f52800b3d00 2024-09-17T03:15:23.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.678+0000 7f5267fff640 1 -- 172.21.15.87:0/750958590 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5270003850 con 0x7f52800b3d00 2024-09-17T03:15:23.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.678+0000 7f5285d58640 1 -- 172.21.15.87:0/750958590 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52801f6210 con 0x7f52800b3d00 2024-09-17T03:15:23.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.679+0000 7f5285d58640 1 -- 172.21.15.87:0/750958590 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52801f45f0 con 0x7f52800b3d00 2024-09-17T03:15:23.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.679+0000 7f5267fff640 1 -- 172.21.15.87:0/750958590 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f527001a070 con 0x7f52800b3d00 2024-09-17T03:15:23.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.680+0000 7f5267fff640 1 --2- 172.21.15.87:0/750958590 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f526c078b90 0x7f526c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:23.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.680+0000 7f5267fff640 1 -- 172.21.15.87:0/750958590 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7f527009a180 con 0x7f52800b3d00 2024-09-17T03:15:23.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.680+0000 7f527effd640 1 --2- 172.21.15.87:0/750958590 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f526c078b90 0x7f526c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:23.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.681+0000 7f527effd640 1 --2- 172.21.15.87:0/750958590 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f526c078b90 0x7f526c07b0a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5274033000 tx=0x7f52740046c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:23.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.685+0000 7f5267fff640 1 -- 172.21.15.87:0/750958590 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f52801f45f0 con 0x7f52800b3d00 2024-09-17T03:15:23.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.859+0000 7f5285d58640 1 -- 172.21.15.87:0/750958590 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52800d1640 con 0x7f52800b3d00 2024-09-17T03:15:23.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.860+0000 7f5267fff640 1 -- 172.21.15.87:0/750958590 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+18525 (secure 0 0 0) 0x7f52800b4650 con 0x7f52800b3d00 2024-09-17T03:15:23.859 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:23.859 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":37,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:15:09.574476+0000","last_up_change":"2024-09-17T03:14:01.833605+0000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:14:03.433404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:14:53.459693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T03:14:54.645011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-09-17T03:15:07.319575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6825","nonce":2430409749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6827","nonce":2430409749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6831","nonce":2430409749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6829","nonce":2430409749}]},"public_addr":"172.21.15.87:6825/2430409749","cluster_addr":"172.21.15.87:6827/2430409749","heartbeat_back_addr":"172.21.15.87:6831/2430409749","heartbeat_front_addr":"172.21.15.87:6829/2430409749","state":["exists","up"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6809","nonce":3467605435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6811","nonce":3467605435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6815","nonce":3467605435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6813","nonce":3467605435}]},"public_addr":"172.21.15.87:6809/3467605435","cluster_addr":"172.21.15.87:6811/3467605435","heartbeat_back_addr":"172.21.15.87:6815/3467605435","heartbeat_front_addr":"172.21.15.87:6813/3467605435","state":["exists","up"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6801","nonce":1450834766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6803","nonce":1450834766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6807","nonce":1450834766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6805","nonce":1450834766}]},"public_addr":"172.21.15.87:6801/1450834766","cluster_addr":"172.21.15.87:6803/1450834766","heartbeat_back_addr":"172.21.15.87:6807/1450834766","heartbeat_front_addr":"172.21.15.87:6805/1450834766","state":["exists","up"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6817","nonce":4253161216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6819","nonce":4253161216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6823","nonce":4253161216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6821","nonce":4253161216}]},"public_addr":"172.21.15.87:6817/4253161216","cluster_addr":"172.21.15.87:6819/4253161216","heartbeat_back_addr":"172.21.15.87:6823/4253161216","heartbeat_front_addr":"172.21.15.87:6821/4253161216","state":["exists","up"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6803","nonce":2414907089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6805","nonce":2414907089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6809","nonce":2414907089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6807","nonce":2414907089}]},"public_addr":"172.21.15.149:6803/2414907089","cluster_addr":"172.21.15.149:6805/2414907089","heartbeat_back_addr":"172.21.15.149:6809/2414907089","heartbeat_front_addr":"172.21.15.149:6807/2414907089","state":["exists","up"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6827","nonce":2182999530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6829","nonce":2182999530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6833","nonce":2182999530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6831","nonce":2182999530}]},"public_addr":"172.21.15.149:6827/2182999530","cluster_addr":"172.21.15.149:6829/2182999530","heartbeat_back_addr":"172.21.15.149:6833/2182999530","heartbeat_front_addr":"172.21.15.149:6831/2182999530","state":["exists","up"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6811","nonce":4289519146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6813","nonce":4289519146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6817","nonce":4289519146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6815","nonce":4289519146}]},"public_addr":"172.21.15.149:6811/4289519146","cluster_addr":"172.21.15.149:6813/4289519146","heartbeat_back_addr":"172.21.15.149:6817/4289519146","heartbeat_front_addr":"172.21.15.149:6815/4289519146","state":["exists","up"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6819","nonce":2771010477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6821","nonce":2771010477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6825","nonce":2771010477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6823","nonce":2771010477}]},"public_addr":"172.21.15.149:6819/2771010477","cluster_addr":"172.21.15.149:6821/2771010477","heartbeat_back_addr":"172.21.15.149:6825/2771010477","heartbeat_front_addr":"172.21.15.149:6823/2771010477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.313297+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.642861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.654781+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.302606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.385791+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.915266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.121365+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.394432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:15:23.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.862+0000 7f5285d58640 1 -- 172.21.15.87:0/750958590 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f526c078b90 msgr2=0x7f526c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:23.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.862+0000 7f5285d58640 1 --2- 172.21.15.87:0/750958590 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f526c078b90 0x7f526c07b0a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5274033000 tx=0x7f52740046c0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.862+0000 7f5285d58640 1 -- 172.21.15.87:0/750958590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52800b3d00 msgr2=0x7f528014e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:23.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.862+0000 7f5285d58640 1 --2- 172.21.15.87:0/750958590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52800b3d00 0x7f528014e870 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f527000c3a0 tx=0x7f527000c860 comp rx=0 tx=0).stop 2024-09-17T03:15:23.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.862+0000 7f5285d58640 1 -- 172.21.15.87:0/750958590 shutdown_connections 2024-09-17T03:15:23.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.862+0000 7f5285d58640 1 --2- 172.21.15.87:0/750958590 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f526c078b90 0x7f526c07b0a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.862+0000 7f5285d58640 1 --2- 172.21.15.87:0/750958590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f528015c6c0 0x7f52800c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.862+0000 7f5285d58640 1 --2- 172.21.15.87:0/750958590 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f52800d0460 0x7f52801f35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.862+0000 7f5285d58640 1 --2- 172.21.15.87:0/750958590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52800b3d00 0x7f528014e870 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.862+0000 7f5285d58640 1 -- 172.21.15.87:0/750958590 >> 172.21.15.87:0/750958590 conn(0x7f5280069360 msgr2=0x7f52800c6230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:23.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.863+0000 7f5285d58640 1 -- 172.21.15.87:0/750958590 shutdown_connections 2024-09-17T03:15:23.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.863+0000 7f5285d58640 1 -- 172.21.15.87:0/750958590 wait complete. 2024-09-17T03:15:23.873 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T03:15:23.873 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:15:23.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.987+0000 7f4884e84640 1 -- 172.21.15.87:0/2723439360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f488015c2d0 msgr2=0x7f48800cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:23.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.987+0000 7f4884e84640 1 --2- 172.21.15.87:0/2723439360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f488015c2d0 0x7f48800cf840 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4870005fd0 tx=0x7f487002a8b0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.987+0000 7f4884e84640 1 -- 172.21.15.87:0/2723439360 shutdown_connections 2024-09-17T03:15:23.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.987+0000 7f4884e84640 1 --2- 172.21.15.87:0/2723439360 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f48800b6600 0x7f488015f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.987+0000 7f4884e84640 1 --2- 172.21.15.87:0/2723439360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f488015c2d0 0x7f48800cf840 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.987+0000 7f4884e84640 1 -- 172.21.15.87:0/2723439360 >> 172.21.15.87:0/2723439360 conn(0x7f48800d4470 msgr2=0x7f48800d4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:23.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.987+0000 7f4884e84640 1 -- 172.21.15.87:0/2723439360 shutdown_connections 2024-09-17T03:15:23.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.987+0000 7f4884e84640 1 -- 172.21.15.87:0/2723439360 wait complete. 2024-09-17T03:15:23.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.989+0000 7f4884e84640 1 Processor -- start 2024-09-17T03:15:23.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.989+0000 7f4884e84640 1 -- start start 2024-09-17T03:15:23.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.990+0000 7f4884e84640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f488015c2d0 0x7f48801f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:23.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.990+0000 7f4884e84640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f488015ecf0 0x7f488015a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:23.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.990+0000 7f4884e84640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48800c8c00 0x7f4880160e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:23.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.990+0000 7f4884e84640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f48800d2db0 con 0x7f48800c8c00 2024-09-17T03:15:23.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.990+0000 7f4884e84640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f48800d2c30 con 0x7f488015c2d0 2024-09-17T03:15:23.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.990+0000 7f4884e84640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f48800d2f30 con 0x7f488015ecf0 2024-09-17T03:15:23.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.990+0000 7f487ed9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48800c8c00 0x7f4880160e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:23.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.990+0000 7f487ed9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48800c8c00 0x7f4880160e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35008/0 (socket says 172.21.15.87:35008) 2024-09-17T03:15:23.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.990+0000 7f487ed9d640 1 -- 172.21.15.87:0/3773891675 learned_addr learned my addr 172.21.15.87:0/3773891675 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:23.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.990+0000 7f487dd9b640 1 --2- 172.21.15.87:0/3773891675 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f488015ecf0 0x7f488015a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:23.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.991+0000 7f487e59c640 1 --2- 172.21.15.87:0/3773891675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f488015c2d0 0x7f48801f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:23.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.991+0000 7f487ed9d640 1 -- 172.21.15.87:0/3773891675 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f488015ecf0 msgr2=0x7f488015a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:23.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.991+0000 7f487ed9d640 1 --2- 172.21.15.87:0/3773891675 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f488015ecf0 0x7f488015a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.991+0000 7f487ed9d640 1 -- 172.21.15.87:0/3773891675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f488015c2d0 msgr2=0x7f48801f5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:23.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.991+0000 7f487ed9d640 1 --2- 172.21.15.87:0/3773891675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f488015c2d0 0x7f48801f5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:23.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.991+0000 7f487ed9d640 1 -- 172.21.15.87:0/3773891675 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48800c9db0 con 0x7f48800c8c00 2024-09-17T03:15:23.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.991+0000 7f487dd9b640 1 --2- 172.21.15.87:0/3773891675 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f488015ecf0 0x7f488015a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:23.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.991+0000 7f487ed9d640 1 --2- 172.21.15.87:0/3773891675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48800c8c00 0x7f4880160e60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f487800d660 tx=0x7f487800db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:23.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.991+0000 7f486effd640 1 -- 172.21.15.87:0/3773891675 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878017070 con 0x7f48800c8c00 2024-09-17T03:15:23.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.991+0000 7f486effd640 1 -- 172.21.15.87:0/3773891675 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4878005310 con 0x7f48800c8c00 2024-09-17T03:15:23.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.991+0000 7f4884e84640 1 -- 172.21.15.87:0/3773891675 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48800c8120 con 0x7f48800c8c00 2024-09-17T03:15:23.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.991+0000 7f486effd640 1 -- 172.21.15.87:0/3773891675 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4878009820 con 0x7f48800c8c00 2024-09-17T03:15:23.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.991+0000 7f4884e84640 1 -- 172.21.15.87:0/3773891675 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48800cc0a0 con 0x7f48800c8c00 2024-09-17T03:15:23.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.992+0000 7f4884e84640 1 -- 172.21.15.87:0/3773891675 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48800cc300 con 0x7f48800c8c00 2024-09-17T03:15:23.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.993+0000 7f486effd640 1 -- 172.21.15.87:0/3773891675 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f48780099c0 con 0x7f48800c8c00 2024-09-17T03:15:23.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.993+0000 7f486effd640 1 --2- 172.21.15.87:0/3773891675 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4860078b90 0x7f486007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:23.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.993+0000 7f486effd640 1 -- 172.21.15.87:0/3773891675 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7f487809aef0 con 0x7f48800c8c00 2024-09-17T03:15:23.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.993+0000 7f487e59c640 1 --2- 172.21.15.87:0/3773891675 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4860078b90 0x7f486007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:23.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.994+0000 7f487e59c640 1 --2- 172.21.15.87:0/3773891675 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4860078b90 0x7f486007b0a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4870034000 tx=0x7f4870004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:23.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:23.998+0000 7f486effd640 1 -- 172.21.15.87:0/3773891675 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f48780632c0 con 0x7f48800c8c00 2024-09-17T03:15:24.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.206+0000 7f4884e84640 1 -- 172.21.15.87:0/3773891675 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48800d3450 con 0x7f48800c8c00 2024-09-17T03:15:24.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.207+0000 7f486effd640 1 -- 172.21.15.87:0/3773891675 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5228 (secure 0 0 0) 0x7f4878062bc0 con 0x7f48800c8c00 2024-09-17T03:15:24.206 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:24.206 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":14,"btime":"2024-09-17T03:15:08:564652+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6836/1081794135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1081794135},{"type":"v1","addr":"172.21.15.149:6836","nonce":1081794135}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2152665509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6835","nonce":2152665509},{"type":"v1","addr":"172.21.15.149:6837","nonce":2152665509}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/704821751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":704821751},{"type":"v1","addr":"172.21.15.87:6835","nonce":704821751}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:14:55.675616+0000","modified":"2024-09-17T03:15:07.917703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5004},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5004":{"gid":5004,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6834/1243349323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":1243349323},{"type":"v1","addr":"172.21.15.87:6834","nonce":1243349323}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5004,"qdb_cluster":[5004]},"id":2}]} 2024-09-17T03:15:24.206 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 14 2024-09-17T03:15:24.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.209+0000 7f4884e84640 1 -- 172.21.15.87:0/3773891675 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4860078b90 msgr2=0x7f486007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:24.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.209+0000 7f4884e84640 1 --2- 172.21.15.87:0/3773891675 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4860078b90 0x7f486007b0a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4870034000 tx=0x7f4870004ab0 comp rx=0 tx=0).stop 2024-09-17T03:15:24.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.209+0000 7f4884e84640 1 -- 172.21.15.87:0/3773891675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48800c8c00 msgr2=0x7f4880160e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:24.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.209+0000 7f4884e84640 1 --2- 172.21.15.87:0/3773891675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48800c8c00 0x7f4880160e60 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f487800d660 tx=0x7f487800db20 comp rx=0 tx=0).stop 2024-09-17T03:15:24.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.209+0000 7f4884e84640 1 -- 172.21.15.87:0/3773891675 shutdown_connections 2024-09-17T03:15:24.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.209+0000 7f4884e84640 1 --2- 172.21.15.87:0/3773891675 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4860078b90 0x7f486007b0a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:24.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.209+0000 7f4884e84640 1 --2- 172.21.15.87:0/3773891675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f48800c8c00 0x7f4880160e60 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:24.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.209+0000 7f4884e84640 1 --2- 172.21.15.87:0/3773891675 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f488015ecf0 0x7f488015a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:24.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.209+0000 7f4884e84640 1 --2- 172.21.15.87:0/3773891675 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f488015c2d0 0x7f48801f5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:24.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.209+0000 7f4884e84640 1 -- 172.21.15.87:0/3773891675 >> 172.21.15.87:0/3773891675 conn(0x7f48800d4470 msgr2=0x7f48800d4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:24.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.209+0000 7f4884e84640 1 -- 172.21.15.87:0/3773891675 shutdown_connections 2024-09-17T03:15:24.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.209+0000 7f4884e84640 1 -- 172.21.15.87:0/3773891675 wait complete. 2024-09-17T03:15:24.219 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:15:24.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.333+0000 7f3d2cba8640 1 -- 172.21.15.87:0/2423710510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d10004800 msgr2=0x7f3d10004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:24.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.333+0000 7f3d2cba8640 1 --2- 172.21.15.87:0/2423710510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d10004800 0x7f3d10004c10 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00c260 tx=0x7f3d10005830 comp rx=0 tx=0).stop 2024-09-17T03:15:24.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.333+0000 7f3d26d9d640 1 -- 172.21.15.87:0/2423710510 reap_dead start 2024-09-17T03:15:24.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.333+0000 7f3d2cba8640 1 -- 172.21.15.87:0/2423710510 shutdown_connections 2024-09-17T03:15:24.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.333+0000 7f3d2cba8640 1 -- 172.21.15.87:0/2423710510 >> 172.21.15.87:0/2423710510 conn(0x7f3d280d1180 msgr2=0x7f3d280d1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:24.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.333+0000 7f3d2cba8640 1 -- 172.21.15.87:0/2423710510 shutdown_connections 2024-09-17T03:15:24.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.333+0000 7f3d2cba8640 1 -- 172.21.15.87:0/2423710510 wait complete. 2024-09-17T03:15:24.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.335+0000 7f3d2cba8640 1 Processor -- start 2024-09-17T03:15:24.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.335+0000 7f3d2cba8640 1 -- start start 2024-09-17T03:15:24.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.336+0000 7f3d2cba8640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3d10004800 0x7f3d281f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:24.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.336+0000 7f3d2cba8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d28154ee0 0x7f3d280aac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:24.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.336+0000 7f3d2cba8640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d281552f0 0x7f3d280ab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:24.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.336+0000 7f3d2cba8640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2815c7c0 con 0x7f3d28154ee0 2024-09-17T03:15:24.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.336+0000 7f3d2cba8640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2815c640 con 0x7f3d281552f0 2024-09-17T03:15:24.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.336+0000 7f3d2cba8640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3d28073e80 con 0x7f3d10004800 2024-09-17T03:15:24.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.336+0000 7f3d25d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d28154ee0 0x7f3d280aac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:24.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.336+0000 7f3d25d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d28154ee0 0x7f3d280aac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35024/0 (socket says 172.21.15.87:35024) 2024-09-17T03:15:24.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.336+0000 7f3d25d9b640 1 -- 172.21.15.87:0/1083147284 learned_addr learned my addr 172.21.15.87:0/1083147284 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:24.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.337+0000 7f3d25d9b640 1 -- 172.21.15.87:0/1083147284 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3d10004800 msgr2=0x7f3d281f4e00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T03:15:24.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.337+0000 7f3d25d9b640 1 --2- 172.21.15.87:0/1083147284 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3d10004800 0x7f3d281f4e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:24.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.337+0000 7f3d25d9b640 1 -- 172.21.15.87:0/1083147284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d281552f0 msgr2=0x7f3d280ab1d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-17T03:15:24.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.337+0000 7f3d25d9b640 1 --2- 172.21.15.87:0/1083147284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d281552f0 0x7f3d280ab1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:24.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.337+0000 7f3d25d9b640 1 -- 172.21.15.87:0/1083147284 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d281f1d20 con 0x7f3d28154ee0 2024-09-17T03:15:24.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.337+0000 7f3d25d9b640 1 --2- 172.21.15.87:0/1083147284 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d28154ee0 0x7f3d280aac90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00c260 tx=0x7f3d1c001d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:24.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.337+0000 7f3d16ffd640 1 -- 172.21.15.87:0/1083147284 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c031040 con 0x7f3d28154ee0 2024-09-17T03:15:24.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.337+0000 7f3d16ffd640 1 -- 172.21.15.87:0/1083147284 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d1c003610 con 0x7f3d28154ee0 2024-09-17T03:15:24.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.337+0000 7f3d2cba8640 1 -- 172.21.15.87:0/1083147284 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d281f0f60 con 0x7f3d28154ee0 2024-09-17T03:15:24.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.337+0000 7f3d16ffd640 1 -- 172.21.15.87:0/1083147284 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c003a40 con 0x7f3d28154ee0 2024-09-17T03:15:24.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.337+0000 7f3d2cba8640 1 -- 172.21.15.87:0/1083147284 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d281f1520 con 0x7f3d28154ee0 2024-09-17T03:15:24.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.338+0000 7f3d2cba8640 1 -- 172.21.15.87:0/1083147284 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d2815a4d0 con 0x7f3d28154ee0 2024-09-17T03:15:24.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.339+0000 7f3d16ffd640 1 -- 172.21.15.87:0/1083147284 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3d1c034070 con 0x7f3d28154ee0 2024-09-17T03:15:24.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.339+0000 7f3d16ffd640 1 --2- 172.21.15.87:0/1083147284 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3d08078b90 0x7f3d0807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:24.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.339+0000 7f3d16ffd640 1 -- 172.21.15.87:0/1083147284 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7f3d1c0bae70 con 0x7f3d28154ee0 2024-09-17T03:15:24.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.339+0000 7f3d2659c640 1 --2- 172.21.15.87:0/1083147284 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3d08078b90 0x7f3d0807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:24.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.340+0000 7f3d2659c640 1 --2- 172.21.15.87:0/1083147284 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3d08078b90 0x7f3d0807b0a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3d18005fd0 tx=0x7f3d18009650 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:24.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.344+0000 7f3d16ffd640 1 -- 172.21.15.87:0/1083147284 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3d1c083200 con 0x7f3d28154ee0 2024-09-17T03:15:24.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.551+0000 7f3d2cba8640 1 -- 172.21.15.87:0/1083147284 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d2815efd0 con 0x7f3d28154ee0 2024-09-17T03:15:24.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.551+0000 7f3d16ffd640 1 -- 172.21.15.87:0/1083147284 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) ==== 94+0+5228 (secure 0 0 0) 0x7f3d1c082b00 con 0x7f3d28154ee0 2024-09-17T03:15:24.550 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:24.550 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":14,"btime":"2024-09-17T03:15:08:564652+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6836/1081794135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1081794135},{"type":"v1","addr":"172.21.15.149:6836","nonce":1081794135}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2152665509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6835","nonce":2152665509},{"type":"v1","addr":"172.21.15.149:6837","nonce":2152665509}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/704821751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":704821751},{"type":"v1","addr":"172.21.15.87:6835","nonce":704821751}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:14:55.675616+0000","modified":"2024-09-17T03:15:07.917703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5004},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5004":{"gid":5004,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6834/1243349323","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":1243349323},{"type":"v1","addr":"172.21.15.87:6834","nonce":1243349323}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5004,"qdb_cluster":[5004]},"id":2}]} 2024-09-17T03:15:24.550 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 14 2024-09-17T03:15:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.553+0000 7f3d2cba8640 1 -- 172.21.15.87:0/1083147284 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3d08078b90 msgr2=0x7f3d0807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.553+0000 7f3d2cba8640 1 --2- 172.21.15.87:0/1083147284 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3d08078b90 0x7f3d0807b0a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3d18005fd0 tx=0x7f3d18009650 comp rx=0 tx=0).stop 2024-09-17T03:15:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.553+0000 7f3d2cba8640 1 -- 172.21.15.87:0/1083147284 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d28154ee0 msgr2=0x7f3d280aac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.553+0000 7f3d2cba8640 1 --2- 172.21.15.87:0/1083147284 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d28154ee0 0x7f3d280aac90 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00c260 tx=0x7f3d1c001d60 comp rx=0 tx=0).stop 2024-09-17T03:15:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.554+0000 7f3d2cba8640 1 -- 172.21.15.87:0/1083147284 shutdown_connections 2024-09-17T03:15:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.554+0000 7f3d2cba8640 1 --2- 172.21.15.87:0/1083147284 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3d08078b90 0x7f3d0807b0a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.554+0000 7f3d2cba8640 1 --2- 172.21.15.87:0/1083147284 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3d281552f0 0x7f3d280ab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.554+0000 7f3d2cba8640 1 --2- 172.21.15.87:0/1083147284 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d28154ee0 0x7f3d280aac90 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.554+0000 7f3d2cba8640 1 --2- 172.21.15.87:0/1083147284 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3d10004800 0x7f3d281f4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.554+0000 7f3d2cba8640 1 -- 172.21.15.87:0/1083147284 >> 172.21.15.87:0/1083147284 conn(0x7f3d280d1180 msgr2=0x7f3d2815d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.554+0000 7f3d2cba8640 1 -- 172.21.15.87:0/1083147284 shutdown_connections 2024-09-17T03:15:24.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.554+0000 7f3d2cba8640 1 -- 172.21.15.87:0/1083147284 wait complete. 2024-09-17T03:15:24.564 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:15:24.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.679+0000 7faa6a2de640 1 -- 172.21.15.87:0/766941991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa640cc590 msgr2=0x7faa6415c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:24.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.679+0000 7faa6a2de640 1 --2- 172.21.15.87:0/766941991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa640cc590 0x7faa6415c470 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7faa54009830 tx=0x7faa5402a510 comp rx=0 tx=0).stop 2024-09-17T03:15:24.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.679+0000 7faa6a2de640 1 -- 172.21.15.87:0/766941991 shutdown_connections 2024-09-17T03:15:24.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.679+0000 7faa6a2de640 1 --2- 172.21.15.87:0/766941991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa640cc590 0x7faa6415c470 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:24.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.679+0000 7faa6a2de640 1 -- 172.21.15.87:0/766941991 >> 172.21.15.87:0/766941991 conn(0x7faa6406ee70 msgr2=0x7faa6406f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:24.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.679+0000 7faa6a2de640 1 -- 172.21.15.87:0/766941991 shutdown_connections 2024-09-17T03:15:24.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.679+0000 7faa6a2de640 1 -- 172.21.15.87:0/766941991 wait complete. 2024-09-17T03:15:24.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.681+0000 7faa6a2de640 1 Processor -- start 2024-09-17T03:15:24.680 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.681+0000 7faa6a2de640 1 -- start start 2024-09-17T03:15:24.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.682+0000 7faa6a2de640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa640cc590 0x7faa640c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:24.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.682+0000 7faa6a2de640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa6415f0c0 0x7faa64156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:24.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.682+0000 7faa6a2de640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7faa640c9a70 0x7faa640c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:24.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.682+0000 7faa6a2de640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7faa6415f650 con 0x7faa640cc590 2024-09-17T03:15:24.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.682+0000 7faa6a2de640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7faa6415f4d0 con 0x7faa6415f0c0 2024-09-17T03:15:24.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.682+0000 7faa6a2de640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7faa6415f7d0 con 0x7faa640c9a70 2024-09-17T03:15:24.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.682+0000 7faa63fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa640cc590 0x7faa640c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:24.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.682+0000 7faa63fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa640cc590 0x7faa640c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35032/0 (socket says 172.21.15.87:35032) 2024-09-17T03:15:24.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.682+0000 7faa63fff640 1 -- 172.21.15.87:0/889074071 learned_addr learned my addr 172.21.15.87:0/889074071 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:24.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.682+0000 7faa6887b640 1 --2- 172.21.15.87:0/889074071 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7faa640c9a70 0x7faa640c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:24.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.683+0000 7faa63fff640 1 -- 172.21.15.87:0/889074071 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7faa640c9a70 msgr2=0x7faa640c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:24.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.683+0000 7faa63fff640 1 --2- 172.21.15.87:0/889074071 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7faa640c9a70 0x7faa640c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:24.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.683+0000 7faa63fff640 1 -- 172.21.15.87:0/889074071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa6415f0c0 msgr2=0x7faa64156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:24.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.683+0000 7faa63fff640 1 --2- 172.21.15.87:0/889074071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa6415f0c0 0x7faa64156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:24.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.683+0000 7faa63fff640 1 -- 172.21.15.87:0/889074071 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa640ca3c0 con 0x7faa640cc590 2024-09-17T03:15:24.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.683+0000 7faa6887b640 1 --2- 172.21.15.87:0/889074071 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7faa640c9a70 0x7faa640c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:24.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.683+0000 7faa63fff640 1 --2- 172.21.15.87:0/889074071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa640cc590 0x7faa640c70d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7faa54000c00 tx=0x7faa54033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:24.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.683+0000 7faa60ff9640 1 -- 172.21.15.87:0/889074071 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa54031d60 con 0x7faa640cc590 2024-09-17T03:15:24.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.683+0000 7faa60ff9640 1 -- 172.21.15.87:0/889074071 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa540330d0 con 0x7faa640cc590 2024-09-17T03:15:24.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.683+0000 7faa6a2de640 1 -- 172.21.15.87:0/889074071 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa640b1cb0 con 0x7faa640cc590 2024-09-17T03:15:24.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.683+0000 7faa60ff9640 1 -- 172.21.15.87:0/889074071 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa54033520 con 0x7faa640cc590 2024-09-17T03:15:24.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.683+0000 7faa6a2de640 1 -- 172.21.15.87:0/889074071 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa640b2270 con 0x7faa640cc590 2024-09-17T03:15:24.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.684+0000 7faa6a2de640 1 -- 172.21.15.87:0/889074071 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa640c5940 con 0x7faa640cc590 2024-09-17T03:15:24.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.685+0000 7faa60ff9640 1 -- 172.21.15.87:0/889074071 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faa540035f0 con 0x7faa640cc590 2024-09-17T03:15:24.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.686+0000 7faa60ff9640 1 --2- 172.21.15.87:0/889074071 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7faa50078bc0 0x7faa5007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:24.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.686+0000 7faa60ff9640 1 -- 172.21.15.87:0/889074071 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7faa540bc360 con 0x7faa640cc590 2024-09-17T03:15:24.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.686+0000 7faa637fe640 1 --2- 172.21.15.87:0/889074071 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7faa50078bc0 0x7faa5007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:24.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.686+0000 7faa637fe640 1 --2- 172.21.15.87:0/889074071 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7faa50078bc0 0x7faa5007b0d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faa58009650 tx=0x7faa58006ba0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:24.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.690+0000 7faa60ff9640 1 -- 172.21.15.87:0/889074071 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7faa640ca3c0 con 0x7faa640cc590 2024-09-17T03:15:24.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.866+0000 7faa6a2de640 1 -- 172.21.15.87:0/889074071 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa64156cc0 con 0x7faa640cc590 2024-09-17T03:15:25.027 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.867+0000 7faa60ff9640 1 -- 172.21.15.87:0/889074071 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+18525 (secure 0 0 0) 0x7faa5403f030 con 0x7faa640cc590 2024-09-17T03:15:25.028 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:25.028 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":37,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:15:09.574476+0000","last_up_change":"2024-09-17T03:14:01.833605+0000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:14:03.433404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:14:53.459693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-17T03:14:54.645011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"data_new","create_time":"2024-09-17T03:15:07.319575+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6825","nonce":2430409749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6827","nonce":2430409749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6831","nonce":2430409749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6829","nonce":2430409749}]},"public_addr":"172.21.15.87:6825/2430409749","cluster_addr":"172.21.15.87:6827/2430409749","heartbeat_back_addr":"172.21.15.87:6831/2430409749","heartbeat_front_addr":"172.21.15.87:6829/2430409749","state":["exists","up"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6809","nonce":3467605435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6811","nonce":3467605435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6815","nonce":3467605435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6813","nonce":3467605435}]},"public_addr":"172.21.15.87:6809/3467605435","cluster_addr":"172.21.15.87:6811/3467605435","heartbeat_back_addr":"172.21.15.87:6815/3467605435","heartbeat_front_addr":"172.21.15.87:6813/3467605435","state":["exists","up"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6801","nonce":1450834766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6803","nonce":1450834766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6807","nonce":1450834766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6805","nonce":1450834766}]},"public_addr":"172.21.15.87:6801/1450834766","cluster_addr":"172.21.15.87:6803/1450834766","heartbeat_back_addr":"172.21.15.87:6807/1450834766","heartbeat_front_addr":"172.21.15.87:6805/1450834766","state":["exists","up"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6817","nonce":4253161216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6819","nonce":4253161216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6823","nonce":4253161216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6821","nonce":4253161216}]},"public_addr":"172.21.15.87:6817/4253161216","cluster_addr":"172.21.15.87:6819/4253161216","heartbeat_back_addr":"172.21.15.87:6823/4253161216","heartbeat_front_addr":"172.21.15.87:6821/4253161216","state":["exists","up"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6803","nonce":2414907089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6805","nonce":2414907089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6809","nonce":2414907089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6807","nonce":2414907089}]},"public_addr":"172.21.15.149:6803/2414907089","cluster_addr":"172.21.15.149:6805/2414907089","heartbeat_back_addr":"172.21.15.149:6809/2414907089","heartbeat_front_addr":"172.21.15.149:6807/2414907089","state":["exists","up"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6827","nonce":2182999530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6829","nonce":2182999530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6833","nonce":2182999530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6831","nonce":2182999530}]},"public_addr":"172.21.15.149:6827/2182999530","cluster_addr":"172.21.15.149:6829/2182999530","heartbeat_back_addr":"172.21.15.149:6833/2182999530","heartbeat_front_addr":"172.21.15.149:6831/2182999530","state":["exists","up"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6811","nonce":4289519146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6813","nonce":4289519146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6817","nonce":4289519146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6815","nonce":4289519146}]},"public_addr":"172.21.15.149:6811/4289519146","cluster_addr":"172.21.15.149:6813/4289519146","heartbeat_back_addr":"172.21.15.149:6817/4289519146","heartbeat_front_addr":"172.21.15.149:6815/4289519146","state":["exists","up"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6819","nonce":2771010477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6821","nonce":2771010477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6825","nonce":2771010477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6823","nonce":2771010477}]},"public_addr":"172.21.15.149:6819/2771010477","cluster_addr":"172.21.15.149:6821/2771010477","heartbeat_back_addr":"172.21.15.149:6825/2771010477","heartbeat_front_addr":"172.21.15.149:6823/2771010477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.313297+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.642861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.654781+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.302606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.385791+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.915266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.121365+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.394432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:15:25.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.869+0000 7faa6a2de640 1 -- 172.21.15.87:0/889074071 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7faa50078bc0 msgr2=0x7faa5007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:25.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.869+0000 7faa6a2de640 1 --2- 172.21.15.87:0/889074071 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7faa50078bc0 0x7faa5007b0d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faa58009650 tx=0x7faa58006ba0 comp rx=0 tx=0).stop 2024-09-17T03:15:25.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.869+0000 7faa6a2de640 1 -- 172.21.15.87:0/889074071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa640cc590 msgr2=0x7faa640c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:25.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.869+0000 7faa6a2de640 1 --2- 172.21.15.87:0/889074071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa640cc590 0x7faa640c70d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7faa54000c00 tx=0x7faa54033d60 comp rx=0 tx=0).stop 2024-09-17T03:15:25.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.869+0000 7faa6a2de640 1 -- 172.21.15.87:0/889074071 shutdown_connections 2024-09-17T03:15:25.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.869+0000 7faa6a2de640 1 --2- 172.21.15.87:0/889074071 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7faa50078bc0 0x7faa5007b0d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:25.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.869+0000 7faa6a2de640 1 --2- 172.21.15.87:0/889074071 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7faa640c9a70 0x7faa640c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:25.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.869+0000 7faa6a2de640 1 --2- 172.21.15.87:0/889074071 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7faa6415f0c0 0x7faa64156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:25.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.869+0000 7faa6a2de640 1 --2- 172.21.15.87:0/889074071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa640cc590 0x7faa640c70d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:25.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.869+0000 7faa6a2de640 1 -- 172.21.15.87:0/889074071 >> 172.21.15.87:0/889074071 conn(0x7faa6406ee70 msgr2=0x7faa64073f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:25.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.869+0000 7faa6a2de640 1 -- 172.21.15.87:0/889074071 shutdown_connections 2024-09-17T03:15:25.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:24.869+0000 7faa6a2de640 1 -- 172.21.15.87:0/889074071 wait complete. 2024-09-17T03:15:25.032 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T03:15:25.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.147+0000 7fb0f5c64640 1 -- 172.21.15.87:0/2116472383 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb0f00cefb0 msgr2=0x7fb0f00cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:25.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.147+0000 7fb0f5c64640 1 --2- 172.21.15.87:0/2116472383 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb0f00cefb0 0x7fb0f00cf3c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8005fd0 tx=0x7fb0e802a540 comp rx=0 tx=0).stop 2024-09-17T03:15:25.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.147+0000 7fb0f5c64640 1 -- 172.21.15.87:0/2116472383 shutdown_connections 2024-09-17T03:15:25.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.147+0000 7fb0f5c64640 1 --2- 172.21.15.87:0/2116472383 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb0f00cefb0 0x7fb0f00cf3c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:25.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.147+0000 7fb0f5c64640 1 --2- 172.21.15.87:0/2116472383 >> v2:172.21.15.149:3300/0 conn(0x7fb0f00d2c50 0x7fb0f00d3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:25.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.147+0000 7fb0f5c64640 1 -- 172.21.15.87:0/2116472383 >> 172.21.15.87:0/2116472383 conn(0x7fb0f008dae0 msgr2=0x7fb0f009d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:25.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.147+0000 7fb0f5c64640 1 -- 172.21.15.87:0/2116472383 shutdown_connections 2024-09-17T03:15:25.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.147+0000 7fb0f5c64640 1 -- 172.21.15.87:0/2116472383 wait complete. 2024-09-17T03:15:25.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.148+0000 7fb0f5c64640 1 Processor -- start 2024-09-17T03:15:25.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.149+0000 7fb0f5c64640 1 -- start start 2024-09-17T03:15:25.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.150+0000 7fb0f5c64640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb0f00cefb0 0x7fb0f01f97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:25.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.150+0000 7fb0f5c64640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0f00d2c50 0x7fb0f015b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:25.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.150+0000 7fb0f5c64640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0f00d4150 0x7fb0f00d10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:25.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.150+0000 7fb0f5c64640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f00cfc90 con 0x7fb0f00d2c50 2024-09-17T03:15:25.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.150+0000 7fb0f5c64640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f00cfb10 con 0x7fb0f00d4150 2024-09-17T03:15:25.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.150+0000 7fb0f5c64640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb0f00cfe10 con 0x7fb0f00cefb0 2024-09-17T03:15:25.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.150+0000 7fb0eeffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0f00d2c50 0x7fb0f015b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:25.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.150+0000 7fb0eeffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0f00d2c50 0x7fb0f015b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35038/0 (socket says 172.21.15.87:35038) 2024-09-17T03:15:25.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.150+0000 7fb0eeffd640 1 -- 172.21.15.87:0/2925719099 learned_addr learned my addr 172.21.15.87:0/2925719099 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:25.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.150+0000 7fb0effff640 1 --2- 172.21.15.87:0/2925719099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0f00d4150 0x7fb0f00d10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:25.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.150+0000 7fb0ef7fe640 1 --2- 172.21.15.87:0/2925719099 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb0f00cefb0 0x7fb0f01f97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:25.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.150+0000 7fb0eeffd640 1 -- 172.21.15.87:0/2925719099 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb0f00cefb0 msgr2=0x7fb0f01f97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:25.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.150+0000 7fb0eeffd640 1 --2- 172.21.15.87:0/2925719099 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb0f00cefb0 0x7fb0f01f97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:25.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.150+0000 7fb0eeffd640 1 -- 172.21.15.87:0/2925719099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0f00d4150 msgr2=0x7fb0f00d10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:25.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.150+0000 7fb0eeffd640 1 --2- 172.21.15.87:0/2925719099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0f00d4150 0x7fb0f00d10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:25.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.150+0000 7fb0eeffd640 1 -- 172.21.15.87:0/2925719099 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f01f9d10 con 0x7fb0f00d2c50 2024-09-17T03:15:25.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.150+0000 7fb0effff640 1 --2- 172.21.15.87:0/2925719099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0f00d4150 0x7fb0f00d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:15:25.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.150+0000 7fb0eeffd640 1 --2- 172.21.15.87:0/2925719099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0f00d2c50 0x7fb0f015b860 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb0d800c430 tx=0x7fb0d800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:25.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.151+0000 7fb0e7fff640 1 -- 172.21.15.87:0/2925719099 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8017070 con 0x7fb0f00d2c50 2024-09-17T03:15:25.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.151+0000 7fb0e7fff640 1 -- 172.21.15.87:0/2925719099 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb0d80053c0 con 0x7fb0f00d2c50 2024-09-17T03:15:25.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.151+0000 7fb0f5c64640 1 -- 172.21.15.87:0/2925719099 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f015bda0 con 0x7fb0f00d2c50 2024-09-17T03:15:25.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.151+0000 7fb0e7fff640 1 -- 172.21.15.87:0/2925719099 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8003850 con 0x7fb0f00d2c50 2024-09-17T03:15:25.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.151+0000 7fb0f5c64640 1 -- 172.21.15.87:0/2925719099 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f00a0080 con 0x7fb0f00d2c50 2024-09-17T03:15:25.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.152+0000 7fb0f5c64640 1 -- 172.21.15.87:0/2925719099 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0f006b140 con 0x7fb0f00d2c50 2024-09-17T03:15:25.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.152+0000 7fb0e7fff640 1 -- 172.21.15.87:0/2925719099 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb0d801a070 con 0x7fb0f00d2c50 2024-09-17T03:15:25.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.153+0000 7fb0e7fff640 1 --2- 172.21.15.87:0/2925719099 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb0d0078b90 0x7fb0d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:25.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.153+0000 7fb0e7fff640 1 -- 172.21.15.87:0/2925719099 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6104+0+0 (secure 0 0 0) 0x7fb0d809a200 con 0x7fb0f00d2c50 2024-09-17T03:15:25.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.153+0000 7fb0ef7fe640 1 --2- 172.21.15.87:0/2925719099 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb0d0078b90 0x7fb0d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:25.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.154+0000 7fb0ef7fe640 1 --2- 172.21.15.87:0/2925719099 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb0d0078b90 0x7fb0d007b0a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0005fd0 tx=0x7fb0e0009500 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:25.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.158+0000 7fb0e7fff640 1 -- 172.21.15.87:0/2925719099 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb0f01f9d10 con 0x7fb0f00d2c50 2024-09-17T03:15:25.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.365+0000 7fb0f5c64640 1 -- 172.21.15.87:0/2925719099 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb0f00d02c0 con 0x7fb0f00d2c50 2024-09-17T03:15:25.365 INFO:tasks.ceph.mon.a.smithi087.stderr:2024-09-17T03:15:25.366+0000 7fafd01f9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T03:15:25.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.367+0000 7fb0e7fff640 1 -- 172.21.15.87:0/2925719099 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v15) ==== 161+0+0 (secure 0 0 0) 0x7fb0d80625d0 con 0x7fb0f00d2c50 2024-09-17T03:15:25.367 INFO:teuthology.orchestra.run.smithi087.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T03:15:25.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.370+0000 7fb0f5c64640 1 -- 172.21.15.87:0/2925719099 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb0d0078b90 msgr2=0x7fb0d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:25.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.370+0000 7fb0f5c64640 1 --2- 172.21.15.87:0/2925719099 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb0d0078b90 0x7fb0d007b0a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0005fd0 tx=0x7fb0e0009500 comp rx=0 tx=0).stop 2024-09-17T03:15:25.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.370+0000 7fb0f5c64640 1 -- 172.21.15.87:0/2925719099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0f00d2c50 msgr2=0x7fb0f015b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:25.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.370+0000 7fb0f5c64640 1 --2- 172.21.15.87:0/2925719099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0f00d2c50 0x7fb0f015b860 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb0d800c430 tx=0x7fb0d800c8f0 comp rx=0 tx=0).stop 2024-09-17T03:15:25.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.370+0000 7fb0f5c64640 1 -- 172.21.15.87:0/2925719099 shutdown_connections 2024-09-17T03:15:25.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.370+0000 7fb0f5c64640 1 --2- 172.21.15.87:0/2925719099 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb0d0078b90 0x7fb0d007b0a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:25.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.370+0000 7fb0f5c64640 1 --2- 172.21.15.87:0/2925719099 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb0f00d4150 0x7fb0f00d10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:25.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.370+0000 7fb0f5c64640 1 --2- 172.21.15.87:0/2925719099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0f00d2c50 0x7fb0f015b860 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:25.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.370+0000 7fb0f5c64640 1 --2- 172.21.15.87:0/2925719099 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb0f00cefb0 0x7fb0f01f97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:25.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.370+0000 7fb0f5c64640 1 -- 172.21.15.87:0/2925719099 >> 172.21.15.87:0/2925719099 conn(0x7fb0f008dae0 msgr2=0x7fb0f0161720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:25.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.370+0000 7fb0f5c64640 1 -- 172.21.15.87:0/2925719099 shutdown_connections 2024-09-17T03:15:25.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.370+0000 7fb0f5c64640 1 -- 172.21.15.87:0/2925719099 wait complete. 2024-09-17T03:15:25.371 INFO:tasks.ceph.mds.c.smithi087.stderr: -4684> 2024-09-17T03:14:49.296+0000 7f18d6332640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-09-17T03:15:25.371 INFO:tasks.ceph.mds.c.smithi087.stderr: -3333> 2024-09-17T03:15:00.144+0000 7f18d0326640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T03:15:25.382 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T03:15:25.426 INFO:tasks.ceph.mds.c.smithi087.stdout:starting mds.c at 2024-09-17T03:15:25.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.500+0000 7f9ede582640 1 -- 172.21.15.87:0/3517016582 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ed80b3080 msgr2=0x7f9ed80cfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:25.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.500+0000 7f9ede582640 1 --2- 172.21.15.87:0/3517016582 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ed80b3080 0x7f9ed80cfe60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f9ed808d740 tx=0x7f9ec802a540 comp rx=0 tx=0).stop 2024-09-17T03:15:25.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.500+0000 7f9ede582640 1 -- 172.21.15.87:0/3517016582 shutdown_connections 2024-09-17T03:15:25.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.500+0000 7f9ede582640 1 --2- 172.21.15.87:0/3517016582 >> v2:172.21.15.149:3300/0 conn(0x7f9ed80d4350 0x7f9ed80d4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:25.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.500+0000 7f9ede582640 1 --2- 172.21.15.87:0/3517016582 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ed80b3080 0x7f9ed80cfe60 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:25.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.500+0000 7f9ede582640 1 -- 172.21.15.87:0/3517016582 >> 172.21.15.87:0/3517016582 conn(0x7f9ed808bfc0 msgr2=0x7f9ed809d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:25.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.500+0000 7f9ede582640 1 -- 172.21.15.87:0/3517016582 shutdown_connections 2024-09-17T03:15:25.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.500+0000 7f9ede582640 1 -- 172.21.15.87:0/3517016582 wait complete. 2024-09-17T03:15:25.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.502+0000 7f9ede582640 1 Processor -- start 2024-09-17T03:15:25.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.502+0000 7f9ede582640 1 -- start start 2024-09-17T03:15:25.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.503+0000 7f9ede582640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ed80b3080 0x7f9ed81518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:25.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.503+0000 7f9ede582640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ed80b3e00 0x7f9ed80ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:25.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.503+0000 7f9ede582640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ed80d4350 0x7f9ed80cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:25.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.503+0000 7f9ede582640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed80d29e0 con 0x7f9ed80b3e00 2024-09-17T03:15:25.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.503+0000 7f9ede582640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9ed80d2860 con 0x7f9ed80b3080 2024-09-17T03:15:25.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.503+0000 7f9ede582640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9ed80d2b60 con 0x7f9ed80d4350 2024-09-17T03:15:25.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.503+0000 7f9ed77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ed80b3e00 0x7f9ed80ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:25.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.503+0000 7f9ed77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ed80b3e00 0x7f9ed80ca400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53468/0 (socket says 172.21.15.87:53468) 2024-09-17T03:15:25.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.503+0000 7f9ed77fe640 1 -- 172.21.15.87:0/1159900239 learned_addr learned my addr 172.21.15.87:0/1159900239 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:25.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.504+0000 7f9ed7fff640 1 --2- 172.21.15.87:0/1159900239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ed80b3080 0x7f9ed81518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:25.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.504+0000 7f9edcb1f640 1 --2- 172.21.15.87:0/1159900239 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ed80d4350 0x7f9ed80cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:25.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.504+0000 7f9ed77fe640 1 -- 172.21.15.87:0/1159900239 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ed80d4350 msgr2=0x7f9ed80cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:25.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.504+0000 7f9ed77fe640 1 --2- 172.21.15.87:0/1159900239 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ed80d4350 0x7f9ed80cd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:25.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.504+0000 7f9ed77fe640 1 -- 172.21.15.87:0/1159900239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ed80b3080 msgr2=0x7f9ed81518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:25.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.504+0000 7f9ed77fe640 1 --2- 172.21.15.87:0/1159900239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ed80b3080 0x7f9ed81518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:25.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.504+0000 7f9ed77fe640 1 -- 172.21.15.87:0/1159900239 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed81f9a30 con 0x7f9ed80b3e00 2024-09-17T03:15:25.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.504+0000 7f9ed77fe640 1 --2- 172.21.15.87:0/1159900239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ed80b3e00 0x7f9ed80ca400 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9ed000cb40 tx=0x7f9ed000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:25.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.504+0000 7f9ed4ff9640 1 -- 172.21.15.87:0/1159900239 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0016020 con 0x7f9ed80b3e00 2024-09-17T03:15:25.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.504+0000 7f9ed4ff9640 1 -- 172.21.15.87:0/1159900239 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ed000ad90 con 0x7f9ed80b3e00 2024-09-17T03:15:25.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.504+0000 7f9ede582640 1 -- 172.21.15.87:0/1159900239 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ed80cc7d0 con 0x7f9ed80b3e00 2024-09-17T03:15:25.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.504+0000 7f9ed4ff9640 1 -- 172.21.15.87:0/1159900239 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed0012680 con 0x7f9ed80b3e00 2024-09-17T03:15:25.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.504+0000 7f9ede582640 1 -- 172.21.15.87:0/1159900239 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ed80ca940 con 0x7f9ed80b3e00 2024-09-17T03:15:25.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.505+0000 7f9ede582640 1 -- 172.21.15.87:0/1159900239 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ed80cc5c0 con 0x7f9ed80b3e00 2024-09-17T03:15:25.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.506+0000 7f9ed4ff9640 1 -- 172.21.15.87:0/1159900239 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9ed0019020 con 0x7f9ed80b3e00 2024-09-17T03:15:25.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.507+0000 7f9ed4ff9640 1 --2- 172.21.15.87:0/1159900239 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9eb8078bc0 0x7f9eb807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:25.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.507+0000 7f9ed4ff9640 1 -- 172.21.15.87:0/1159900239 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6190+0+0 (secure 0 0 0) 0x7f9ed009b180 con 0x7f9ed80b3e00 2024-09-17T03:15:25.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.507+0000 7f9ed7fff640 1 --2- 172.21.15.87:0/1159900239 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9eb8078bc0 0x7f9eb807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:25.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.507+0000 7f9ed7fff640 1 --2- 172.21.15.87:0/1159900239 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9eb8078bc0 0x7f9eb807b0d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8034000 tx=0x7f9ec8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:25.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.511+0000 7f9ed4ff9640 1 -- 172.21.15.87:0/1159900239 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f9ed0020030 con 0x7f9ed80b3e00 2024-09-17T03:15:25.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:25.722+0000 7f9ede582640 1 -- 172.21.15.87:0/1159900239 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9ed81f97c0 con 0x7f9ed80b3e00 2024-09-17T03:15:26.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.375+0000 7f9ed4ff9640 1 -- 172.21.15.87:0/1159900239 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v16) ==== 104+0+0 (secure 0 0 0) 0x7f9ed0062d10 con 0x7f9ed80b3e00 2024-09-17T03:15:26.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.377+0000 7f9ede582640 1 -- 172.21.15.87:0/1159900239 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9eb8078bc0 msgr2=0x7f9eb807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:26.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.377+0000 7f9ede582640 1 --2- 172.21.15.87:0/1159900239 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9eb8078bc0 0x7f9eb807b0d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8034000 tx=0x7f9ec8004730 comp rx=0 tx=0).stop 2024-09-17T03:15:26.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.377+0000 7f9ede582640 1 -- 172.21.15.87:0/1159900239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ed80b3e00 msgr2=0x7f9ed80ca400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:26.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.377+0000 7f9ede582640 1 --2- 172.21.15.87:0/1159900239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ed80b3e00 0x7f9ed80ca400 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9ed000cb40 tx=0x7f9ed000a060 comp rx=0 tx=0).stop 2024-09-17T03:15:26.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.377+0000 7f9ede582640 1 -- 172.21.15.87:0/1159900239 shutdown_connections 2024-09-17T03:15:26.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.377+0000 7f9ede582640 1 --2- 172.21.15.87:0/1159900239 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9eb8078bc0 0x7f9eb807b0d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:26.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.377+0000 7f9ede582640 1 --2- 172.21.15.87:0/1159900239 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f9ed80d4350 0x7f9ed80cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:26.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.377+0000 7f9ede582640 1 --2- 172.21.15.87:0/1159900239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ed80b3e00 0x7f9ed80ca400 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:26.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.377+0000 7f9ede582640 1 --2- 172.21.15.87:0/1159900239 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f9ed80b3080 0x7f9ed81518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:26.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.377+0000 7f9ede582640 1 -- 172.21.15.87:0/1159900239 >> 172.21.15.87:0/1159900239 conn(0x7f9ed808bfc0 msgr2=0x7f9ed80b5200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:26.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.377+0000 7f9ede582640 1 -- 172.21.15.87:0/1159900239 shutdown_connections 2024-09-17T03:15:26.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.377+0000 7f9ede582640 1 -- 172.21.15.87:0/1159900239 wait complete. 2024-09-17T03:15:26.390 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T03:15:26.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.508+0000 7f35c6094640 1 -- 172.21.15.87:0/3913203204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35ac004d90 msgr2=0x7f35ac005260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:26.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.508+0000 7f35c6094640 1 --2- 172.21.15.87:0/3913203204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35ac004d90 0x7f35ac005260 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f35c00d21f0 tx=0x7f35ac005830 comp rx=0 tx=0).stop 2024-09-17T03:15:26.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.508+0000 7f35bffff640 1 -- 172.21.15.87:0/3913203204 reap_dead start 2024-09-17T03:15:26.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.508+0000 7f35c6094640 1 -- 172.21.15.87:0/3913203204 shutdown_connections 2024-09-17T03:15:26.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.508+0000 7f35c6094640 1 -- 172.21.15.87:0/3913203204 >> 172.21.15.87:0/3913203204 conn(0x7f35c00b2cc0 msgr2=0x7f35c00b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:26.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.508+0000 7f35c6094640 1 -- 172.21.15.87:0/3913203204 shutdown_connections 2024-09-17T03:15:26.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.508+0000 7f35c6094640 1 -- 172.21.15.87:0/3913203204 wait complete. 2024-09-17T03:15:26.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.509+0000 7f35c6094640 1 Processor -- start 2024-09-17T03:15:26.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.510+0000 7f35c6094640 1 -- start start 2024-09-17T03:15:26.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.511+0000 7f35c6094640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35ac004d90 0x7f35c01f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:26.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.511+0000 7f35c6094640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35c01f78e0 0x7f35c009bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:26.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.511+0000 7f35c6094640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35c009c090 0x7f35c01536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:26.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.511+0000 7f35c6094640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f35c0070100 con 0x7f35c01f78e0 2024-09-17T03:15:26.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.511+0000 7f35c6094640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f35c006ff80 con 0x7f35c009c090 2024-09-17T03:15:26.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.511+0000 7f35c6094640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f35c0164780 con 0x7f35ac004d90 2024-09-17T03:15:26.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.511+0000 7f35beffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35c01f78e0 0x7f35c009bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:26.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.511+0000 7f35beffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35c01f78e0 0x7f35c009bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53486/0 (socket says 172.21.15.87:53486) 2024-09-17T03:15:26.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.511+0000 7f35beffd640 1 -- 172.21.15.87:0/1563827155 learned_addr learned my addr 172.21.15.87:0/1563827155 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:26.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.511+0000 7f35beffd640 1 -- 172.21.15.87:0/1563827155 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35ac004d90 msgr2=0x7f35c01f73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:26.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.511+0000 7f35beffd640 1 --2- 172.21.15.87:0/1563827155 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35ac004d90 0x7f35c01f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:26.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.511+0000 7f35beffd640 1 -- 172.21.15.87:0/1563827155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35c009c090 msgr2=0x7f35c01536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:26.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.511+0000 7f35beffd640 1 --2- 172.21.15.87:0/1563827155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35c009c090 0x7f35c01536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:26.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.511+0000 7f35beffd640 1 -- 172.21.15.87:0/1563827155 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c009c4a0 con 0x7f35c01f78e0 2024-09-17T03:15:26.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.512+0000 7f35beffd640 1 --2- 172.21.15.87:0/1563827155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35c01f78e0 0x7f35c009bb50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f35a800e590 tx=0x7f35a800ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:26.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.512+0000 7f35b7fff640 1 -- 172.21.15.87:0/1563827155 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a8015070 con 0x7f35c01f78e0 2024-09-17T03:15:26.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.512+0000 7f35b7fff640 1 -- 172.21.15.87:0/1563827155 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35a80109a0 con 0x7f35c01f78e0 2024-09-17T03:15:26.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.512+0000 7f35b7fff640 1 -- 172.21.15.87:0/1563827155 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a8010df0 con 0x7f35c01f78e0 2024-09-17T03:15:26.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.512+0000 7f35c6094640 1 -- 172.21.15.87:0/1563827155 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35c01f3800 con 0x7f35c01f78e0 2024-09-17T03:15:26.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.512+0000 7f35c6094640 1 -- 172.21.15.87:0/1563827155 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35c01f3da0 con 0x7f35c01f78e0 2024-09-17T03:15:26.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.513+0000 7f35c6094640 1 -- 172.21.15.87:0/1563827155 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35c00ce250 con 0x7f35c01f78e0 2024-09-17T03:15:26.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.513+0000 7f35b7fff640 1 -- 172.21.15.87:0/1563827155 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f35a8018070 con 0x7f35c01f78e0 2024-09-17T03:15:26.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.514+0000 7f35b7fff640 1 --2- 172.21.15.87:0/1563827155 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f35a0078bc0 0x7f35a007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:26.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.514+0000 7f35b7fff640 1 -- 172.21.15.87:0/1563827155 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6190+0+0 (secure 0 0 0) 0x7f35a809b300 con 0x7f35c01f78e0 2024-09-17T03:15:26.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.514+0000 7f35bf7fe640 1 --2- 172.21.15.87:0/1563827155 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f35a0078bc0 0x7f35a007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:26.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.515+0000 7f35bf7fe640 1 --2- 172.21.15.87:0/1563827155 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f35a0078bc0 0x7f35a007b0d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f35b0006fd0 tx=0x7f35b0006920 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:26.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.518+0000 7f35b7fff640 1 -- 172.21.15.87:0/1563827155 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f35c009c4a0 con 0x7f35c01f78e0 2024-09-17T03:15:26.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:26.695+0000 7f35c6094640 1 -- 172.21.15.87:0/1563827155 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f35c00cdc90 con 0x7f35c01f78e0 2024-09-17T03:15:27.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.658+0000 7f35b7fff640 1 -- 172.21.15.87:0/1563827155 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v39) ==== 181+0+0 (secure 0 0 0) 0x7f35a80635f0 con 0x7f35c01f78e0 2024-09-17T03:15:27.658 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' removed 2024-09-17T03:15:27.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.661+0000 7f35c6094640 1 -- 172.21.15.87:0/1563827155 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f35a0078bc0 msgr2=0x7f35a007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:27.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.661+0000 7f35c6094640 1 --2- 172.21.15.87:0/1563827155 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f35a0078bc0 0x7f35a007b0d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f35b0006fd0 tx=0x7f35b0006920 comp rx=0 tx=0).stop 2024-09-17T03:15:27.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.661+0000 7f35c6094640 1 -- 172.21.15.87:0/1563827155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35c01f78e0 msgr2=0x7f35c009bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:27.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.661+0000 7f35c6094640 1 --2- 172.21.15.87:0/1563827155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35c01f78e0 0x7f35c009bb50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f35a800e590 tx=0x7f35a800ea50 comp rx=0 tx=0).stop 2024-09-17T03:15:27.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.661+0000 7f35c6094640 1 -- 172.21.15.87:0/1563827155 shutdown_connections 2024-09-17T03:15:27.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.661+0000 7f35c6094640 1 --2- 172.21.15.87:0/1563827155 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f35a0078bc0 0x7f35a007b0d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:27.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.661+0000 7f35c6094640 1 --2- 172.21.15.87:0/1563827155 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f35c009c090 0x7f35c01536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:27.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.661+0000 7f35c6094640 1 --2- 172.21.15.87:0/1563827155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35c01f78e0 0x7f35c009bb50 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:27.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.661+0000 7f35c6094640 1 --2- 172.21.15.87:0/1563827155 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f35ac004d90 0x7f35c01f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:27.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.661+0000 7f35c6094640 1 -- 172.21.15.87:0/1563827155 >> 172.21.15.87:0/1563827155 conn(0x7f35c00b2cc0 msgr2=0x7f35c00b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:27.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.661+0000 7f35c6094640 1 -- 172.21.15.87:0/1563827155 shutdown_connections 2024-09-17T03:15:27.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.661+0000 7f35c6094640 1 -- 172.21.15.87:0/1563827155 wait complete. 2024-09-17T03:15:27.676 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T03:15:27.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.795+0000 7f0a30d82640 1 -- 172.21.15.87:0/3425499771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a14003a50 msgr2=0x7f0a14003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:27.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.795+0000 7f0a30d82640 1 --2- 172.21.15.87:0/3425499771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a14003a50 0x7f0a14003ec0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c009b20 tx=0x7f0a14005830 comp rx=0 tx=0).stop 2024-09-17T03:15:27.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.796+0000 7f0a30d82640 1 -- 172.21.15.87:0/3425499771 shutdown_connections 2024-09-17T03:15:27.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.796+0000 7f0a30d82640 1 --2- 172.21.15.87:0/3425499771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a14003a50 0x7f0a14003ec0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:27.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.796+0000 7f0a30d82640 1 -- 172.21.15.87:0/3425499771 >> 172.21.15.87:0/3425499771 conn(0x7f0a2c0b2cc0 msgr2=0x7f0a2c0b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:27.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.796+0000 7f0a30d82640 1 -- 172.21.15.87:0/3425499771 shutdown_connections 2024-09-17T03:15:27.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.796+0000 7f0a30d82640 1 -- 172.21.15.87:0/3425499771 wait complete. 2024-09-17T03:15:27.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.797+0000 7f0a30d82640 1 Processor -- start 2024-09-17T03:15:27.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.798+0000 7f0a30d82640 1 -- start start 2024-09-17T03:15:27.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.799+0000 7f0a30d82640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a14003a50 0x7f0a2c1539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:27.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.799+0000 7f0a30d82640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a2c157fb0 0x7f0a2c15e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:27.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.799+0000 7f0a30d82640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a2c15eac0 0x7f0a2c154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:27.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.799+0000 7f0a30d82640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c074780 con 0x7f0a2c15eac0 2024-09-17T03:15:27.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.799+0000 7f0a30d82640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0a2c074600 con 0x7f0a14003a50 2024-09-17T03:15:27.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.799+0000 7f0a30d82640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0a2c06d1f0 con 0x7f0a2c157fb0 2024-09-17T03:15:27.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.799+0000 7f0a2ad9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a2c15eac0 0x7f0a2c154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:27.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.799+0000 7f0a2ad9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a2c15eac0 0x7f0a2c154910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53494/0 (socket says 172.21.15.87:53494) 2024-09-17T03:15:27.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.799+0000 7f0a2ad9d640 1 -- 172.21.15.87:0/2363574109 learned_addr learned my addr 172.21.15.87:0/2363574109 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:27.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.799+0000 7f0a2a59c640 1 --2- 172.21.15.87:0/2363574109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a14003a50 0x7f0a2c1539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:27.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.799+0000 7f0a29d9b640 1 --2- 172.21.15.87:0/2363574109 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a2c157fb0 0x7f0a2c15e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:27.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.799+0000 7f0a2ad9d640 1 -- 172.21.15.87:0/2363574109 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a2c157fb0 msgr2=0x7f0a2c15e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:27.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.799+0000 7f0a2ad9d640 1 --2- 172.21.15.87:0/2363574109 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a2c157fb0 0x7f0a2c15e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:27.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.799+0000 7f0a2ad9d640 1 -- 172.21.15.87:0/2363574109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a14003a50 msgr2=0x7f0a2c1539c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:27.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.799+0000 7f0a2ad9d640 1 --2- 172.21.15.87:0/2363574109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a14003a50 0x7f0a2c1539c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:27.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.799+0000 7f0a2ad9d640 1 -- 172.21.15.87:0/2363574109 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a2c15eed0 con 0x7f0a2c15eac0 2024-09-17T03:15:27.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.799+0000 7f0a2ad9d640 1 --2- 172.21.15.87:0/2363574109 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a2c15eac0 0x7f0a2c154910 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c069b50 tx=0x7f0a240019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:27.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.800+0000 7f0a1affd640 1 -- 172.21.15.87:0/2363574109 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a24011870 con 0x7f0a2c15eac0 2024-09-17T03:15:27.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.800+0000 7f0a1affd640 1 -- 172.21.15.87:0/2363574109 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a24017070 con 0x7f0a2c15eac0 2024-09-17T03:15:27.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.800+0000 7f0a30d82640 1 -- 172.21.15.87:0/2363574109 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a2c158ea0 con 0x7f0a2c15eac0 2024-09-17T03:15:27.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.800+0000 7f0a1affd640 1 -- 172.21.15.87:0/2363574109 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a24011da0 con 0x7f0a2c15eac0 2024-09-17T03:15:27.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.800+0000 7f0a30d82640 1 -- 172.21.15.87:0/2363574109 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a2c1f33a0 con 0x7f0a2c15eac0 2024-09-17T03:15:27.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.801+0000 7f0a30d82640 1 -- 172.21.15.87:0/2363574109 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a2c153f00 con 0x7f0a2c15eac0 2024-09-17T03:15:27.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.801+0000 7f0a1affd640 1 -- 172.21.15.87:0/2363574109 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0a2401c460 con 0x7f0a2c15eac0 2024-09-17T03:15:27.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.802+0000 7f0a1affd640 1 --2- 172.21.15.87:0/2363574109 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0a0c078b90 0x7f0a0c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:27.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.802+0000 7f0a1affd640 1 -- 172.21.15.87:0/2363574109 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5752+0+0 (secure 0 0 0) 0x7f0a2409b420 con 0x7f0a2c15eac0 2024-09-17T03:15:27.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.802+0000 7f0a2a59c640 1 --2- 172.21.15.87:0/2363574109 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0a0c078b90 0x7f0a0c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:27.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.803+0000 7f0a2a59c640 1 --2- 172.21.15.87:0/2363574109 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0a0c078b90 0x7f0a0c07b0a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c035000 tx=0x7f0a1c0046c0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:27.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.806+0000 7f0a1affd640 1 -- 172.21.15.87:0/2363574109 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0a2c153f00 con 0x7f0a2c15eac0 2024-09-17T03:15:27.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:27.981+0000 7f0a30d82640 1 -- 172.21.15.87:0/2363574109 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0a2c06ef00 con 0x7f0a2c15eac0 2024-09-17T03:15:28.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.666+0000 7f0a1affd640 1 -- 172.21.15.87:0/2363574109 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v40) ==== 169+0+0 (secure 0 0 0) 0x7f0a2c15eed0 con 0x7f0a2c15eac0 2024-09-17T03:15:28.666 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' removed 2024-09-17T03:15:28.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.668+0000 7f0a30d82640 1 -- 172.21.15.87:0/2363574109 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0a0c078b90 msgr2=0x7f0a0c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:28.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.668+0000 7f0a30d82640 1 --2- 172.21.15.87:0/2363574109 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0a0c078b90 0x7f0a0c07b0a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c035000 tx=0x7f0a1c0046c0 comp rx=0 tx=0).stop 2024-09-17T03:15:28.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.668+0000 7f0a30d82640 1 -- 172.21.15.87:0/2363574109 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a2c15eac0 msgr2=0x7f0a2c154910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:28.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.668+0000 7f0a30d82640 1 --2- 172.21.15.87:0/2363574109 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a2c15eac0 0x7f0a2c154910 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c069b50 tx=0x7f0a240019b0 comp rx=0 tx=0).stop 2024-09-17T03:15:28.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.668+0000 7f0a30d82640 1 -- 172.21.15.87:0/2363574109 shutdown_connections 2024-09-17T03:15:28.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.668+0000 7f0a30d82640 1 --2- 172.21.15.87:0/2363574109 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0a0c078b90 0x7f0a0c07b0a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:28.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.668+0000 7f0a30d82640 1 --2- 172.21.15.87:0/2363574109 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a2c15eac0 0x7f0a2c154910 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:28.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.669+0000 7f0a30d82640 1 --2- 172.21.15.87:0/2363574109 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0a2c157fb0 0x7f0a2c15e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:28.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.669+0000 7f0a30d82640 1 --2- 172.21.15.87:0/2363574109 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0a14003a50 0x7f0a2c1539c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:28.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.669+0000 7f0a30d82640 1 -- 172.21.15.87:0/2363574109 >> 172.21.15.87:0/2363574109 conn(0x7f0a2c0b2cc0 msgr2=0x7f0a2c0ae340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:28.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.669+0000 7f0a30d82640 1 -- 172.21.15.87:0/2363574109 shutdown_connections 2024-09-17T03:15:28.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.669+0000 7f0a30d82640 1 -- 172.21.15.87:0/2363574109 wait complete. 2024-09-17T03:15:28.682 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm data_new data_new --yes-i-really-really-mean-it 2024-09-17T03:15:28.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.802+0000 7f4b782ce640 1 -- 172.21.15.87:0/2092780970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b5c004d40 msgr2=0x7f4b5c005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:28.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.802+0000 7f4b782ce640 1 --2- 172.21.15.87:0/2092780970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b5c004d40 0x7f4b5c005150 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800c290 tx=0x7f4b5c005830 comp rx=0 tx=0).stop 2024-09-17T03:15:28.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.802+0000 7f4b7686b640 1 -- 172.21.15.87:0/2092780970 reap_dead start 2024-09-17T03:15:28.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.802+0000 7f4b782ce640 1 -- 172.21.15.87:0/2092780970 shutdown_connections 2024-09-17T03:15:28.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.802+0000 7f4b782ce640 1 -- 172.21.15.87:0/2092780970 >> 172.21.15.87:0/2092780970 conn(0x7f4b70072b00 msgr2=0x7f4b70072f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:28.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.802+0000 7f4b782ce640 1 -- 172.21.15.87:0/2092780970 shutdown_connections 2024-09-17T03:15:28.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.802+0000 7f4b782ce640 1 -- 172.21.15.87:0/2092780970 wait complete. 2024-09-17T03:15:28.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.804+0000 7f4b782ce640 1 Processor -- start 2024-09-17T03:15:28.803 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.804+0000 7f4b782ce640 1 -- start start 2024-09-17T03:15:28.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.805+0000 7f4b782ce640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b5c004d40 0x7f4b701f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:28.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.805+0000 7f4b782ce640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b70157aa0 0x7f4b7006ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:28.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.805+0000 7f4b782ce640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b7006f280 0x7f4b700b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:28.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.805+0000 7f4b782ce640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4b700b51a0 con 0x7f4b5c004d40 2024-09-17T03:15:28.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.805+0000 7f4b782ce640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4b700d1300 con 0x7f4b7006f280 2024-09-17T03:15:28.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.805+0000 7f4b782ce640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4b700b5320 con 0x7f4b70157aa0 2024-09-17T03:15:28.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.805+0000 7f4b7606a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b5c004d40 0x7f4b701f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:28.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.805+0000 7f4b7606a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b5c004d40 0x7f4b701f5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53502/0 (socket says 172.21.15.87:53502) 2024-09-17T03:15:28.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.805+0000 7f4b7606a640 1 -- 172.21.15.87:0/3932572803 learned_addr learned my addr 172.21.15.87:0/3932572803 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:28.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.805+0000 7f4b75869640 1 --2- 172.21.15.87:0/3932572803 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b70157aa0 0x7f4b7006ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:28.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.805+0000 7f4b7686b640 1 --2- 172.21.15.87:0/3932572803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b7006f280 0x7f4b700b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:28.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.805+0000 7f4b7606a640 1 -- 172.21.15.87:0/3932572803 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b70157aa0 msgr2=0x7f4b7006ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:28.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.805+0000 7f4b7606a640 1 --2- 172.21.15.87:0/3932572803 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b70157aa0 0x7f4b7006ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:28.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.805+0000 7f4b7606a640 1 -- 172.21.15.87:0/3932572803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b7006f280 msgr2=0x7f4b700b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:28.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.805+0000 7f4b7606a640 1 --2- 172.21.15.87:0/3932572803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b7006f280 0x7f4b700b0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:28.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.805+0000 7f4b7606a640 1 -- 172.21.15.87:0/3932572803 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b70155d50 con 0x7f4b5c004d40 2024-09-17T03:15:28.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.805+0000 7f4b7686b640 1 --2- 172.21.15.87:0/3932572803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b7006f280 0x7f4b700b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:28.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.806+0000 7f4b7606a640 1 --2- 172.21.15.87:0/3932572803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b5c004d40 0x7f4b701f5fc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c011da0 tx=0x7f4b6c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:28.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.806+0000 7f4b66ffd640 1 -- 172.21.15.87:0/3932572803 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c003d80 con 0x7f4b5c004d40 2024-09-17T03:15:28.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.806+0000 7f4b66ffd640 1 -- 172.21.15.87:0/3932572803 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b6c004b20 con 0x7f4b5c004d40 2024-09-17T03:15:28.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.806+0000 7f4b66ffd640 1 -- 172.21.15.87:0/3932572803 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c004050 con 0x7f4b5c004d40 2024-09-17T03:15:28.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.806+0000 7f4b782ce640 1 -- 172.21.15.87:0/3932572803 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b70155fc0 con 0x7f4b5c004d40 2024-09-17T03:15:28.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.806+0000 7f4b782ce640 1 -- 172.21.15.87:0/3932572803 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b70156580 con 0x7f4b5c004d40 2024-09-17T03:15:28.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.807+0000 7f4b782ce640 1 -- 172.21.15.87:0/3932572803 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b700b7f30 con 0x7f4b5c004d40 2024-09-17T03:15:28.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.808+0000 7f4b66ffd640 1 -- 172.21.15.87:0/3932572803 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4b6c0041f0 con 0x7f4b5c004d40 2024-09-17T03:15:28.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.808+0000 7f4b66ffd640 1 --2- 172.21.15.87:0/3932572803 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4b54078b90 0x7f4b5407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:28.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.808+0000 7f4b66ffd640 1 -- 172.21.15.87:0/3932572803 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 5338+0+0 (secure 0 0 0) 0x7f4b6c09be00 con 0x7f4b5c004d40 2024-09-17T03:15:28.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.808+0000 7f4b75869640 1 --2- 172.21.15.87:0/3932572803 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4b54078b90 0x7f4b5407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:28.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.809+0000 7f4b75869640 1 --2- 172.21.15.87:0/3932572803 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4b54078b90 0x7f4b5407b0a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4b7006fb60 tx=0x7f4b60005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:28.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.813+0000 7f4b66ffd640 1 -- 172.21.15.87:0/3932572803 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4b6c0643b0 con 0x7f4b5c004d40 2024-09-17T03:15:28.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:28.986+0000 7f4b782ce640 1 -- 172.21.15.87:0/3932572803 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4b7015f770 con 0x7f4b5c004d40 2024-09-17T03:15:29.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.671+0000 7f4b66ffd640 1 -- 172.21.15.87:0/3932572803 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "data_new", "pool2": "data_new", "yes_i_really_really_mean_it": true}]=0 pool 'data_new' removed v41) ==== 160+0+0 (secure 0 0 0) 0x7f4b6c063cb0 con 0x7f4b5c004d40 2024-09-17T03:15:29.671 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'data_new' removed 2024-09-17T03:15:29.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.674+0000 7f4b782ce640 1 -- 172.21.15.87:0/3932572803 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4b54078b90 msgr2=0x7f4b5407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:29.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.674+0000 7f4b782ce640 1 --2- 172.21.15.87:0/3932572803 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4b54078b90 0x7f4b5407b0a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4b7006fb60 tx=0x7f4b60005f90 comp rx=0 tx=0).stop 2024-09-17T03:15:29.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.674+0000 7f4b782ce640 1 -- 172.21.15.87:0/3932572803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b5c004d40 msgr2=0x7f4b701f5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:29.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.674+0000 7f4b782ce640 1 --2- 172.21.15.87:0/3932572803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b5c004d40 0x7f4b701f5fc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c011da0 tx=0x7f4b6c00f790 comp rx=0 tx=0).stop 2024-09-17T03:15:29.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.674+0000 7f4b782ce640 1 -- 172.21.15.87:0/3932572803 shutdown_connections 2024-09-17T03:15:29.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.674+0000 7f4b782ce640 1 --2- 172.21.15.87:0/3932572803 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4b54078b90 0x7f4b5407b0a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:29.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.674+0000 7f4b782ce640 1 --2- 172.21.15.87:0/3932572803 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4b7006f280 0x7f4b700b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:29.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.674+0000 7f4b782ce640 1 --2- 172.21.15.87:0/3932572803 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4b70157aa0 0x7f4b7006ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:29.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.674+0000 7f4b782ce640 1 --2- 172.21.15.87:0/3932572803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b5c004d40 0x7f4b701f5fc0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:29.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.674+0000 7f4b782ce640 1 -- 172.21.15.87:0/3932572803 >> 172.21.15.87:0/3932572803 conn(0x7f4b70072b00 msgr2=0x7f4b7015cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:29.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.674+0000 7f4b782ce640 1 -- 172.21.15.87:0/3932572803 shutdown_connections 2024-09-17T03:15:29.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.674+0000 7f4b782ce640 1 -- 172.21.15.87:0/3932572803 wait complete. 2024-09-17T03:15:29.685 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T03:15:29.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.800+0000 7f03be8f9640 1 -- 172.21.15.87:0/2182700387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03b815c6c0 msgr2=0x7f03b8160c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:29.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.800+0000 7f03be8f9640 1 --2- 172.21.15.87:0/2182700387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03b815c6c0 0x7f03b8160c20 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f03b80749b0 tx=0x7f03a802a4b0 comp rx=0 tx=0).stop 2024-09-17T03:15:29.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.800+0000 7f03be8f9640 1 -- 172.21.15.87:0/2182700387 shutdown_connections 2024-09-17T03:15:29.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.800+0000 7f03be8f9640 1 --2- 172.21.15.87:0/2182700387 >> v2:172.21.15.149:3300/0 conn(0x7f03b80b3d00 0x7f03b80b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:29.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.800+0000 7f03be8f9640 1 --2- 172.21.15.87:0/2182700387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03b815c6c0 0x7f03b8160c20 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:29.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.800+0000 7f03be8f9640 1 -- 172.21.15.87:0/2182700387 >> 172.21.15.87:0/2182700387 conn(0x7f03b8069360 msgr2=0x7f03b80ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:29.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.800+0000 7f03be8f9640 1 -- 172.21.15.87:0/2182700387 shutdown_connections 2024-09-17T03:15:29.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.800+0000 7f03be8f9640 1 -- 172.21.15.87:0/2182700387 wait complete. 2024-09-17T03:15:29.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.802+0000 7f03be8f9640 1 Processor -- start 2024-09-17T03:15:29.802 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.802+0000 7f03be8f9640 1 -- start start 2024-09-17T03:15:29.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.804+0000 7f03be8f9640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f03b80710e0 0x7f03b8155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:29.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.804+0000 7f03be8f9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03b80b3d00 0x7f03b806f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:29.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.804+0000 7f03be8f9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03b815c6c0 0x7f03b80c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:29.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.804+0000 7f03be8f9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f03b80b8080 con 0x7f03b80b3d00 2024-09-17T03:15:29.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.804+0000 7f03be8f9640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f03b80b7f00 con 0x7f03b815c6c0 2024-09-17T03:15:29.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.804+0000 7f03be8f9640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f03b80d1470 con 0x7f03b80710e0 2024-09-17T03:15:29.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.804+0000 7f03b77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03b80b3d00 0x7f03b806f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:29.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.804+0000 7f03b77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03b80b3d00 0x7f03b806f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53516/0 (socket says 172.21.15.87:53516) 2024-09-17T03:15:29.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.804+0000 7f03b77fe640 1 -- 172.21.15.87:0/1424099642 learned_addr learned my addr 172.21.15.87:0/1424099642 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:29.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.804+0000 7f03b7fff640 1 --2- 172.21.15.87:0/1424099642 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f03b80710e0 0x7f03b8155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:29.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.804+0000 7f03bce96640 1 --2- 172.21.15.87:0/1424099642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03b815c6c0 0x7f03b80c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:29.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.804+0000 7f03b77fe640 1 -- 172.21.15.87:0/1424099642 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f03b80710e0 msgr2=0x7f03b8155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:29.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.804+0000 7f03b77fe640 1 --2- 172.21.15.87:0/1424099642 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f03b80710e0 0x7f03b8155c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:29.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.804+0000 7f03b77fe640 1 -- 172.21.15.87:0/1424099642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03b815c6c0 msgr2=0x7f03b80c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:29.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.804+0000 7f03b77fe640 1 --2- 172.21.15.87:0/1424099642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03b815c6c0 0x7f03b80c7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:29.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.804+0000 7f03b77fe640 1 -- 172.21.15.87:0/1424099642 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b8156d00 con 0x7f03b80b3d00 2024-09-17T03:15:29.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.804+0000 7f03b77fe640 1 --2- 172.21.15.87:0/1424099642 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03b80b3d00 0x7f03b806f4d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00b440 tx=0x7f03ac00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:29.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.804+0000 7f03b4ff9640 1 -- 172.21.15.87:0/1424099642 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac015020 con 0x7f03b80b3d00 2024-09-17T03:15:29.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.804+0000 7f03b4ff9640 1 -- 172.21.15.87:0/1424099642 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03ac011980 con 0x7f03b80b3d00 2024-09-17T03:15:29.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.804+0000 7f03be8f9640 1 -- 172.21.15.87:0/1424099642 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b81f8c30 con 0x7f03b80b3d00 2024-09-17T03:15:29.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.804+0000 7f03b4ff9640 1 -- 172.21.15.87:0/1424099642 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac011dd0 con 0x7f03b80b3d00 2024-09-17T03:15:29.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.804+0000 7f03be8f9640 1 -- 172.21.15.87:0/1424099642 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b81f91f0 con 0x7f03b80b3d00 2024-09-17T03:15:29.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.806+0000 7f03be8f9640 1 -- 172.21.15.87:0/1424099642 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03b8071ce0 con 0x7f03b80b3d00 2024-09-17T03:15:29.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.806+0000 7f03b4ff9640 1 -- 172.21.15.87:0/1424099642 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f03ac018020 con 0x7f03b80b3d00 2024-09-17T03:15:29.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.807+0000 7f03b4ff9640 1 --2- 172.21.15.87:0/1424099642 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f03a4078b90 0x7f03a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:29.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.807+0000 7f03b4ff9640 1 -- 172.21.15.87:0/1424099642 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f03ac09aab0 con 0x7f03b80b3d00 2024-09-17T03:15:29.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.807+0000 7f03b7fff640 1 --2- 172.21.15.87:0/1424099642 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f03a4078b90 0x7f03a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:29.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.808+0000 7f03b7fff640 1 --2- 172.21.15.87:0/1424099642 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f03a4078b90 0x7f03a407b0a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f03a8033000 tx=0x7f03a8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:29.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.811+0000 7f03b4ff9640 1 -- 172.21.15.87:0/1424099642 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f03ac01f020 con 0x7f03b80b3d00 2024-09-17T03:15:29.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.987+0000 7f03be8f9640 1 -- 172.21.15.87:0/1424099642 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f03b806fdd0 con 0x7f03b80b3d00 2024-09-17T03:15:29.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.991+0000 7f03b4ff9640 1 -- 172.21.15.87:0/1424099642 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v33) ==== 71+0+0 (secure 0 0 0) 0x7f03ac00d040 con 0x7f03b80b3d00 2024-09-17T03:15:29.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.993+0000 7f03be8f9640 1 -- 172.21.15.87:0/1424099642 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f03a4078b90 msgr2=0x7f03a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:29.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.993+0000 7f03be8f9640 1 --2- 172.21.15.87:0/1424099642 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f03a4078b90 0x7f03a407b0a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f03a8033000 tx=0x7f03a8004730 comp rx=0 tx=0).stop 2024-09-17T03:15:29.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.993+0000 7f03be8f9640 1 -- 172.21.15.87:0/1424099642 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03b80b3d00 msgr2=0x7f03b806f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:29.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.993+0000 7f03be8f9640 1 --2- 172.21.15.87:0/1424099642 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03b80b3d00 0x7f03b806f4d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00b440 tx=0x7f03ac00b900 comp rx=0 tx=0).stop 2024-09-17T03:15:29.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.993+0000 7f03be8f9640 1 -- 172.21.15.87:0/1424099642 shutdown_connections 2024-09-17T03:15:29.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.993+0000 7f03be8f9640 1 --2- 172.21.15.87:0/1424099642 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f03a4078b90 0x7f03a407b0a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:29.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.993+0000 7f03be8f9640 1 --2- 172.21.15.87:0/1424099642 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f03b815c6c0 0x7f03b80c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:29.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.993+0000 7f03be8f9640 1 --2- 172.21.15.87:0/1424099642 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03b80b3d00 0x7f03b806f4d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:29.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.993+0000 7f03be8f9640 1 --2- 172.21.15.87:0/1424099642 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f03b80710e0 0x7f03b8155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:29.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.993+0000 7f03be8f9640 1 -- 172.21.15.87:0/1424099642 >> 172.21.15.87:0/1424099642 conn(0x7f03b8069360 msgr2=0x7f03b80714f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:29.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.994+0000 7f03be8f9640 1 -- 172.21.15.87:0/1424099642 shutdown_connections 2024-09-17T03:15:29.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:29.994+0000 7f03be8f9640 1 -- 172.21.15.87:0/1424099642 wait complete. 2024-09-17T03:15:30.005 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification' 2024-09-17T03:15:30.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.117+0000 7f976e9a8640 1 -- 172.21.15.87:0/311157515 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9750004d40 msgr2=0x7f97500051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:30.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.117+0000 7f976e9a8640 1 --2- 172.21.15.87:0/311157515 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9750004d40 0x7f97500051f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9768074cb0 tx=0x7f9750005830 comp rx=0 tx=0).stop 2024-09-17T03:15:30.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.117+0000 7f976cf45640 1 -- 172.21.15.87:0/311157515 reap_dead start 2024-09-17T03:15:30.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.117+0000 7f976e9a8640 1 -- 172.21.15.87:0/311157515 shutdown_connections 2024-09-17T03:15:30.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.117+0000 7f976e9a8640 1 -- 172.21.15.87:0/311157515 >> 172.21.15.87:0/311157515 conn(0x7f97680d2230 msgr2=0x7f97680d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:30.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.117+0000 7f976e9a8640 1 -- 172.21.15.87:0/311157515 shutdown_connections 2024-09-17T03:15:30.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.117+0000 7f976e9a8640 1 -- 172.21.15.87:0/311157515 wait complete. 2024-09-17T03:15:30.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.119+0000 7f976e9a8640 1 Processor -- start 2024-09-17T03:15:30.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.119+0000 7f976e9a8640 1 -- start start 2024-09-17T03:15:30.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.121+0000 7f976e9a8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97681e1260 0x7f976809be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:30.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.121+0000 7f976e9a8640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97681e0a30 0x7f97680cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:30.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.121+0000 7f976e9a8640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97680cdae0 0x7f97680af8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:30.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.121+0000 7f976e9a8640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f97680d13d0 con 0x7f97681e1260 2024-09-17T03:15:30.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.121+0000 7f976e9a8640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f97680d1250 con 0x7f97680cdae0 2024-09-17T03:15:30.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.121+0000 7f976e9a8640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f97680cee00 con 0x7f97681e0a30 2024-09-17T03:15:30.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.121+0000 7f9767fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97681e1260 0x7f976809be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:30.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.121+0000 7f9767fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97681e1260 0x7f976809be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53536/0 (socket says 172.21.15.87:53536) 2024-09-17T03:15:30.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.121+0000 7f9767fff640 1 -- 172.21.15.87:0/2331616696 learned_addr learned my addr 172.21.15.87:0/2331616696 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:30.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.121+0000 7f976cf45640 1 --2- 172.21.15.87:0/2331616696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97680cdae0 0x7f97680af8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:30.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.121+0000 7f97677fe640 1 --2- 172.21.15.87:0/2331616696 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97681e0a30 0x7f97680cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:30.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.121+0000 7f9767fff640 1 -- 172.21.15.87:0/2331616696 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97681e0a30 msgr2=0x7f97680cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:30.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.121+0000 7f9767fff640 1 --2- 172.21.15.87:0/2331616696 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97681e0a30 0x7f97680cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:30.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.121+0000 7f9767fff640 1 -- 172.21.15.87:0/2331616696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97680cdae0 msgr2=0x7f97680af8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:30.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.121+0000 7f9767fff640 1 --2- 172.21.15.87:0/2331616696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97680cdae0 0x7f97680af8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:30.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.121+0000 7f9767fff640 1 -- 172.21.15.87:0/2331616696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97680cdef0 con 0x7f97681e1260 2024-09-17T03:15:30.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.121+0000 7f976cf45640 1 --2- 172.21.15.87:0/2331616696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97680cdae0 0x7f97680af8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:30.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.121+0000 7f9767fff640 1 --2- 172.21.15.87:0/2331616696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97681e1260 0x7f976809be90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9758011a80 tx=0x7f9758011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:30.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.122+0000 7f9764ff9640 1 -- 172.21.15.87:0/2331616696 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975800fe10 con 0x7f97681e1260 2024-09-17T03:15:30.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.122+0000 7f9764ff9640 1 -- 172.21.15.87:0/2331616696 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9758016070 con 0x7f97681e1260 2024-09-17T03:15:30.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.122+0000 7f976e9a8640 1 -- 172.21.15.87:0/2331616696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97681e1a60 con 0x7f97681e1260 2024-09-17T03:15:30.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.122+0000 7f9764ff9640 1 -- 172.21.15.87:0/2331616696 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758012690 con 0x7f97681e1260 2024-09-17T03:15:30.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.122+0000 7f976e9a8640 1 -- 172.21.15.87:0/2331616696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f976809f0c0 con 0x7f97681e1260 2024-09-17T03:15:30.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.123+0000 7f976e9a8640 1 -- 172.21.15.87:0/2331616696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97680cd390 con 0x7f97681e1260 2024-09-17T03:15:30.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.123+0000 7f9764ff9640 1 -- 172.21.15.87:0/2331616696 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9758018030 con 0x7f97681e1260 2024-09-17T03:15:30.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.124+0000 7f9764ff9640 1 --2- 172.21.15.87:0/2331616696 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9754078b90 0x7f975407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:30.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.124+0000 7f9764ff9640 1 -- 172.21.15.87:0/2331616696 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f975809b2a0 con 0x7f97681e1260 2024-09-17T03:15:30.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.124+0000 7f97677fe640 1 --2- 172.21.15.87:0/2331616696 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9754078b90 0x7f975407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:30.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.125+0000 7f97677fe640 1 --2- 172.21.15.87:0/2331616696 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9754078b90 0x7f975407b0a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f975c003670 tx=0x7f975c0067a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:30.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.129+0000 7f9764ff9640 1 -- 172.21.15.87:0/2331616696 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f97680cdef0 con 0x7f97681e1260 2024-09-17T03:15:30.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.374+0000 7f976e9a8640 1 -- 172.21.15.87:0/2331616696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]} v 0) -- 0x7f97680d0d00 con 0x7f97681e1260 2024-09-17T03:15:30.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.673+0000 7f9764ff9640 1 -- 172.21.15.87:0/2331616696 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_forbidden_modification"]}]=0 v90) ==== 147+0+0 (secure 0 0 0) 0x7f9758063b00 con 0x7f97681e1260 2024-09-17T03:15:30.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.675+0000 7f976e9a8640 1 -- 172.21.15.87:0/2331616696 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9754078b90 msgr2=0x7f975407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:30.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.675+0000 7f976e9a8640 1 --2- 172.21.15.87:0/2331616696 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9754078b90 0x7f975407b0a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f975c003670 tx=0x7f975c0067a0 comp rx=0 tx=0).stop 2024-09-17T03:15:30.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.675+0000 7f976e9a8640 1 -- 172.21.15.87:0/2331616696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97681e1260 msgr2=0x7f976809be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:30.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.675+0000 7f976e9a8640 1 --2- 172.21.15.87:0/2331616696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97681e1260 0x7f976809be90 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f9758011a80 tx=0x7f9758011f40 comp rx=0 tx=0).stop 2024-09-17T03:15:30.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.675+0000 7f976e9a8640 1 -- 172.21.15.87:0/2331616696 shutdown_connections 2024-09-17T03:15:30.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.675+0000 7f976e9a8640 1 --2- 172.21.15.87:0/2331616696 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9754078b90 0x7f975407b0a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:30.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.675+0000 7f976e9a8640 1 --2- 172.21.15.87:0/2331616696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f97680cdae0 0x7f97680af8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:30.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.675+0000 7f976e9a8640 1 --2- 172.21.15.87:0/2331616696 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97681e0a30 0x7f97680cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:30.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.675+0000 7f976e9a8640 1 --2- 172.21.15.87:0/2331616696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f97681e1260 0x7f976809be90 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:30.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.675+0000 7f976e9a8640 1 -- 172.21.15.87:0/2331616696 >> 172.21.15.87:0/2331616696 conn(0x7f97680d2230 msgr2=0x7f97681e46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:30.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.675+0000 7f976e9a8640 1 -- 172.21.15.87:0/2331616696 shutdown_connections 2024-09-17T03:15:30.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.675+0000 7f976e9a8640 1 -- 172.21.15.87:0/2331616696 wait complete. 2024-09-17T03:15:30.687 INFO:tasks.cephfs_test_runner:That a client who does not have the capability for setting ... ok 2024-09-17T03:15:30.687 INFO:tasks.cephfs_test_runner:Starting test: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-09-17T03:15:30.687 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:15:30.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.800+0000 7fa3127da640 1 -- 172.21.15.87:0/1931826011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa30c0d0460 msgr2=0x7fa30c0d0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:30.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.800+0000 7fa3127da640 1 --2- 172.21.15.87:0/1931826011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa30c0d0460 0x7fa30c0d0870 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0749b0 tx=0x7fa30402a980 comp rx=0 tx=0).stop 2024-09-17T03:15:30.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.800+0000 7fa3127da640 1 -- 172.21.15.87:0/1931826011 shutdown_connections 2024-09-17T03:15:30.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.800+0000 7fa3127da640 1 --2- 172.21.15.87:0/1931826011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa30c0d0460 0x7fa30c0d0870 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:30.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.800+0000 7fa3127da640 1 --2- 172.21.15.87:0/1931826011 >> v2:172.21.15.149:3300/0 conn(0x7fa30c15c6c0 0x7fa30c160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:30.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.800+0000 7fa3127da640 1 -- 172.21.15.87:0/1931826011 >> 172.21.15.87:0/1931826011 conn(0x7fa30c069360 msgr2=0x7fa30c0ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:30.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.800+0000 7fa3127da640 1 -- 172.21.15.87:0/1931826011 shutdown_connections 2024-09-17T03:15:30.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.800+0000 7fa3127da640 1 -- 172.21.15.87:0/1931826011 wait complete. 2024-09-17T03:15:30.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.802+0000 7fa3127da640 1 Processor -- start 2024-09-17T03:15:30.801 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.802+0000 7fa3127da640 1 -- start start 2024-09-17T03:15:30.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.803+0000 7fa3127da640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa30c0b3d00 0x7fa30c15ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:30.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.803+0000 7fa3127da640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa30c0d0460 0x7fa30c1f3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:30.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.803+0000 7fa3127da640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa30c15c6c0 0x7fa30c0b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:30.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.803+0000 7fa3127da640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c0d0db0 con 0x7fa30c0d0460 2024-09-17T03:15:30.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.803+0000 7fa3127da640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa30c0b80d0 con 0x7fa30c0b3d00 2024-09-17T03:15:30.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.803+0000 7fa3127da640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa30c0d0f30 con 0x7fa30c15c6c0 2024-09-17T03:15:30.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.803+0000 7fa30b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa30c0d0460 0x7fa30c1f3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:30.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.803+0000 7fa30b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa30c0d0460 0x7fa30c1f3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53560/0 (socket says 172.21.15.87:53560) 2024-09-17T03:15:30.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.803+0000 7fa30b7fe640 1 -- 172.21.15.87:0/2254712389 learned_addr learned my addr 172.21.15.87:0/2254712389 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:30.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.803+0000 7fa310d77640 1 --2- 172.21.15.87:0/2254712389 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa30c15c6c0 0x7fa30c0b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:30.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.803+0000 7fa30bfff640 1 --2- 172.21.15.87:0/2254712389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa30c0b3d00 0x7fa30c15ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:30.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.803+0000 7fa30b7fe640 1 -- 172.21.15.87:0/2254712389 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa30c15c6c0 msgr2=0x7fa30c0b5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:30.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.803+0000 7fa30b7fe640 1 --2- 172.21.15.87:0/2254712389 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa30c15c6c0 0x7fa30c0b5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:30.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.803+0000 7fa30b7fe640 1 -- 172.21.15.87:0/2254712389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa30c0b3d00 msgr2=0x7fa30c15ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:30.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.804+0000 7fa30b7fe640 1 --2- 172.21.15.87:0/2254712389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa30c0b3d00 0x7fa30c15ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:30.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.804+0000 7fa30b7fe640 1 -- 172.21.15.87:0/2254712389 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa30c156480 con 0x7fa30c0d0460 2024-09-17T03:15:30.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.804+0000 7fa310d77640 1 --2- 172.21.15.87:0/2254712389 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa30c15c6c0 0x7fa30c0b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:30.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.804+0000 7fa30b7fe640 1 --2- 172.21.15.87:0/2254712389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa30c0d0460 0x7fa30c1f3930 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa30000b420 tx=0x7fa30000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:30.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.804+0000 7fa308ff9640 1 -- 172.21.15.87:0/2254712389 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa300011830 con 0x7fa30c0d0460 2024-09-17T03:15:30.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.804+0000 7fa308ff9640 1 -- 172.21.15.87:0/2254712389 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa300016070 con 0x7fa30c0d0460 2024-09-17T03:15:30.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.804+0000 7fa3127da640 1 -- 172.21.15.87:0/2254712389 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa30c156760 con 0x7fa30c0d0460 2024-09-17T03:15:30.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.804+0000 7fa308ff9640 1 -- 172.21.15.87:0/2254712389 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa300011d60 con 0x7fa30c0d0460 2024-09-17T03:15:30.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.804+0000 7fa3127da640 1 -- 172.21.15.87:0/2254712389 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa30c156d20 con 0x7fa30c0d0460 2024-09-17T03:15:30.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.805+0000 7fa3127da640 1 -- 172.21.15.87:0/2254712389 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa30c0d1450 con 0x7fa30c0d0460 2024-09-17T03:15:30.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.806+0000 7fa308ff9640 1 -- 172.21.15.87:0/2254712389 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa30001b460 con 0x7fa30c0d0460 2024-09-17T03:15:30.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.806+0000 7fa308ff9640 1 --2- 172.21.15.87:0/2254712389 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa2f8078b90 0x7fa2f807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:30.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.806+0000 7fa308ff9640 1 -- 172.21.15.87:0/2254712389 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7fa30009a880 con 0x7fa30c0d0460 2024-09-17T03:15:30.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.806+0000 7fa30bfff640 1 --2- 172.21.15.87:0/2254712389 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa2f8078b90 0x7fa2f807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:30.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.807+0000 7fa30bfff640 1 --2- 172.21.15.87:0/2254712389 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa2f8078b90 0x7fa2f807b0a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc005d70 tx=0x7fa2fc005d00 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:30.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.811+0000 7fa308ff9640 1 -- 172.21.15.87:0/2254712389 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa300062f40 con 0x7fa30c0d0460 2024-09-17T03:15:30.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.984+0000 7fa3127da640 1 -- 172.21.15.87:0/2254712389 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa30c0cef50 con 0x7fa30c0d0460 2024-09-17T03:15:30.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.985+0000 7fa308ff9640 1 -- 172.21.15.87:0/2254712389 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+13056 (secure 0 0 0) 0x7fa300062840 con 0x7fa30c0d0460 2024-09-17T03:15:30.985 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:30.985 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":41,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:15:29.667964+0000","last_up_change":"2024-09-17T03:14:01.833605+0000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:14:03.433404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6825","nonce":2430409749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6827","nonce":2430409749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6831","nonce":2430409749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6829","nonce":2430409749}]},"public_addr":"172.21.15.87:6825/2430409749","cluster_addr":"172.21.15.87:6827/2430409749","heartbeat_back_addr":"172.21.15.87:6831/2430409749","heartbeat_front_addr":"172.21.15.87:6829/2430409749","state":["exists","up"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6809","nonce":3467605435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6811","nonce":3467605435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6815","nonce":3467605435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6813","nonce":3467605435}]},"public_addr":"172.21.15.87:6809/3467605435","cluster_addr":"172.21.15.87:6811/3467605435","heartbeat_back_addr":"172.21.15.87:6815/3467605435","heartbeat_front_addr":"172.21.15.87:6813/3467605435","state":["exists","up"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6801","nonce":1450834766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6803","nonce":1450834766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6807","nonce":1450834766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6805","nonce":1450834766}]},"public_addr":"172.21.15.87:6801/1450834766","cluster_addr":"172.21.15.87:6803/1450834766","heartbeat_back_addr":"172.21.15.87:6807/1450834766","heartbeat_front_addr":"172.21.15.87:6805/1450834766","state":["exists","up"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6817","nonce":4253161216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6819","nonce":4253161216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6823","nonce":4253161216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6821","nonce":4253161216}]},"public_addr":"172.21.15.87:6817/4253161216","cluster_addr":"172.21.15.87:6819/4253161216","heartbeat_back_addr":"172.21.15.87:6823/4253161216","heartbeat_front_addr":"172.21.15.87:6821/4253161216","state":["exists","up"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6803","nonce":2414907089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6805","nonce":2414907089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6809","nonce":2414907089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6807","nonce":2414907089}]},"public_addr":"172.21.15.149:6803/2414907089","cluster_addr":"172.21.15.149:6805/2414907089","heartbeat_back_addr":"172.21.15.149:6809/2414907089","heartbeat_front_addr":"172.21.15.149:6807/2414907089","state":["exists","up"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6827","nonce":2182999530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6829","nonce":2182999530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6833","nonce":2182999530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6831","nonce":2182999530}]},"public_addr":"172.21.15.149:6827/2182999530","cluster_addr":"172.21.15.149:6829/2182999530","heartbeat_back_addr":"172.21.15.149:6833/2182999530","heartbeat_front_addr":"172.21.15.149:6831/2182999530","state":["exists","up"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6811","nonce":4289519146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6813","nonce":4289519146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6817","nonce":4289519146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6815","nonce":4289519146}]},"public_addr":"172.21.15.149:6811/4289519146","cluster_addr":"172.21.15.149:6813/4289519146","heartbeat_back_addr":"172.21.15.149:6817/4289519146","heartbeat_front_addr":"172.21.15.149:6815/4289519146","state":["exists","up"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6819","nonce":2771010477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6821","nonce":2771010477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6825","nonce":2771010477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6823","nonce":2771010477}]},"public_addr":"172.21.15.149:6819/2771010477","cluster_addr":"172.21.15.149:6821/2771010477","heartbeat_back_addr":"172.21.15.149:6825/2771010477","heartbeat_front_addr":"172.21.15.149:6823/2771010477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.313297+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.642861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.654781+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.302606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.385791+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.915266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.121365+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.394432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6834/1243349323":"2024-09-18T03:15:25.367531+0000","172.21.15.87:6833/1243349323":"2024-09-18T03:15:25.367531+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:15:30.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.987+0000 7fa3127da640 1 -- 172.21.15.87:0/2254712389 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa2f8078b90 msgr2=0x7fa2f807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:30.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.987+0000 7fa3127da640 1 --2- 172.21.15.87:0/2254712389 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa2f8078b90 0x7fa2f807b0a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc005d70 tx=0x7fa2fc005d00 comp rx=0 tx=0).stop 2024-09-17T03:15:30.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.987+0000 7fa3127da640 1 -- 172.21.15.87:0/2254712389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa30c0d0460 msgr2=0x7fa30c1f3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:30.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.987+0000 7fa3127da640 1 --2- 172.21.15.87:0/2254712389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa30c0d0460 0x7fa30c1f3930 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa30000b420 tx=0x7fa30000b8e0 comp rx=0 tx=0).stop 2024-09-17T03:15:30.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.987+0000 7fa3127da640 1 -- 172.21.15.87:0/2254712389 shutdown_connections 2024-09-17T03:15:30.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.987+0000 7fa3127da640 1 --2- 172.21.15.87:0/2254712389 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa2f8078b90 0x7fa2f807b0a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:30.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.987+0000 7fa3127da640 1 --2- 172.21.15.87:0/2254712389 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa30c15c6c0 0x7fa30c0b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:30.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.987+0000 7fa3127da640 1 --2- 172.21.15.87:0/2254712389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa30c0d0460 0x7fa30c1f3930 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:30.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.987+0000 7fa3127da640 1 --2- 172.21.15.87:0/2254712389 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa30c0b3d00 0x7fa30c15ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:30.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.987+0000 7fa3127da640 1 -- 172.21.15.87:0/2254712389 >> 172.21.15.87:0/2254712389 conn(0x7fa30c069360 msgr2=0x7fa30c0b6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:30.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.987+0000 7fa3127da640 1 -- 172.21.15.87:0/2254712389 shutdown_connections 2024-09-17T03:15:30.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:30.987+0000 7fa3127da640 1 -- 172.21.15.87:0/2254712389 wait complete. 2024-09-17T03:15:30.998 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-17T03:14:03.433404+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-17T03:15:30.999 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-17T03:15:31.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.113+0000 7fcdf3320640 1 -- 172.21.15.87:0/901275727 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdec15fa20 msgr2=0x7fcdec06cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:31.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.113+0000 7fcdf3320640 1 --2- 172.21.15.87:0/901275727 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdec15fa20 0x7fcdec06cde0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fcdec051960 tx=0x7fcde002a530 comp rx=0 tx=0).stop 2024-09-17T03:15:31.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.113+0000 7fcdf3320640 1 -- 172.21.15.87:0/901275727 shutdown_connections 2024-09-17T03:15:31.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.113+0000 7fcdf3320640 1 --2- 172.21.15.87:0/901275727 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdec15fa20 0x7fcdec06cde0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:31.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.113+0000 7fcdf3320640 1 --2- 172.21.15.87:0/901275727 >> v2:172.21.15.149:3300/0 conn(0x7fcdec071c80 0x7fcdec162a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:31.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.113+0000 7fcdf3320640 1 -- 172.21.15.87:0/901275727 >> 172.21.15.87:0/901275727 conn(0x7fcdec0cdbc0 msgr2=0x7fcdec0cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:31.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.113+0000 7fcdf3320640 1 -- 172.21.15.87:0/901275727 shutdown_connections 2024-09-17T03:15:31.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.113+0000 7fcdf3320640 1 -- 172.21.15.87:0/901275727 wait complete. 2024-09-17T03:15:31.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.115+0000 7fcdf3320640 1 Processor -- start 2024-09-17T03:15:31.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.115+0000 7fcdf3320640 1 -- start start 2024-09-17T03:15:31.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.116+0000 7fcdf3320640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdec071c80 0x7fcdec1e8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:31.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.116+0000 7fcdf3320640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdec15fa20 0x7fcdec139930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:31.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.116+0000 7fcdf3320640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcdec162fd0 0x7fcdec160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:31.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.116+0000 7fcdf3320640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec0cad20 con 0x7fcdec071c80 2024-09-17T03:15:31.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.116+0000 7fcdf3320640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec0caba0 con 0x7fcdec15fa20 2024-09-17T03:15:31.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.116+0000 7fcdf3320640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcdec0caea0 con 0x7fcdec162fd0 2024-09-17T03:15:31.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.116+0000 7fcdf10bc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdec071c80 0x7fcdec1e8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:31.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.116+0000 7fcdf10bc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdec071c80 0x7fcdec1e8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53586/0 (socket says 172.21.15.87:53586) 2024-09-17T03:15:31.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.116+0000 7fcdf10bc640 1 -- 172.21.15.87:0/2857855116 learned_addr learned my addr 172.21.15.87:0/2857855116 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:31.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.117+0000 7fcdf08bb640 1 --2- 172.21.15.87:0/2857855116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdec15fa20 0x7fcdec139930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:31.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.117+0000 7fcdf18bd640 1 --2- 172.21.15.87:0/2857855116 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcdec162fd0 0x7fcdec160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:31.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.117+0000 7fcdf10bc640 1 -- 172.21.15.87:0/2857855116 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcdec162fd0 msgr2=0x7fcdec160fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:31.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.117+0000 7fcdf10bc640 1 --2- 172.21.15.87:0/2857855116 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcdec162fd0 0x7fcdec160fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:31.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.117+0000 7fcdf10bc640 1 -- 172.21.15.87:0/2857855116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdec15fa20 msgr2=0x7fcdec139930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:31.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.117+0000 7fcdf10bc640 1 --2- 172.21.15.87:0/2857855116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdec15fa20 0x7fcdec139930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:31.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.117+0000 7fcdf10bc640 1 -- 172.21.15.87:0/2857855116 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdec1f9120 con 0x7fcdec071c80 2024-09-17T03:15:31.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.117+0000 7fcdf10bc640 1 --2- 172.21.15.87:0/2857855116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdec071c80 0x7fcdec1e8440 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00ea20 tx=0x7fcddc00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:31.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.117+0000 7fcde9ffb640 1 -- 172.21.15.87:0/2857855116 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc00cde0 con 0x7fcdec071c80 2024-09-17T03:15:31.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.117+0000 7fcde9ffb640 1 -- 172.21.15.87:0/2857855116 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcddc011560 con 0x7fcdec071c80 2024-09-17T03:15:31.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.117+0000 7fcdf3320640 1 -- 172.21.15.87:0/2857855116 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdec1f9400 con 0x7fcdec071c80 2024-09-17T03:15:31.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.117+0000 7fcde9ffb640 1 -- 172.21.15.87:0/2857855116 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc011990 con 0x7fcdec071c80 2024-09-17T03:15:31.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.117+0000 7fcdf3320640 1 -- 172.21.15.87:0/2857855116 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdec1f99c0 con 0x7fcdec071c80 2024-09-17T03:15:31.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.119+0000 7fcdf3320640 1 -- 172.21.15.87:0/2857855116 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdec1f9da0 con 0x7fcdec071c80 2024-09-17T03:15:31.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.119+0000 7fcde9ffb640 1 -- 172.21.15.87:0/2857855116 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcddc011b30 con 0x7fcdec071c80 2024-09-17T03:15:31.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.120+0000 7fcde9ffb640 1 --2- 172.21.15.87:0/2857855116 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcdd8078b90 0x7fcdd807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:31.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.120+0000 7fcde9ffb640 1 -- 172.21.15.87:0/2857855116 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7fcddc09b700 con 0x7fcdec071c80 2024-09-17T03:15:31.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.120+0000 7fcdf08bb640 1 --2- 172.21.15.87:0/2857855116 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcdd8078b90 0x7fcdd807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:31.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.120+0000 7fcdf08bb640 1 --2- 172.21.15.87:0/2857855116 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcdd8078b90 0x7fcdd807b0a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcde0034000 tx=0x7fcde0004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:31.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.124+0000 7fcde9ffb640 1 -- 172.21.15.87:0/2857855116 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fcdec1f9da0 con 0x7fcdec071c80 2024-09-17T03:15:31.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.296+0000 7fcdf3320640 1 -- 172.21.15.87:0/2857855116 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcdec162a90 con 0x7fcdec071c80 2024-09-17T03:15:31.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.297+0000 7fcde9ffb640 1 -- 172.21.15.87:0/2857855116 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) ==== 93+0+10 (secure 0 0 0) 0x7fcddc016030 con 0x7fcdec071c80 2024-09-17T03:15:31.297 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-09-17T03:15:31.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.299+0000 7fcdf3320640 1 -- 172.21.15.87:0/2857855116 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcdd8078b90 msgr2=0x7fcdd807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:31.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.299+0000 7fcdf3320640 1 --2- 172.21.15.87:0/2857855116 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcdd8078b90 0x7fcdd807b0a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcde0034000 tx=0x7fcde0004730 comp rx=0 tx=0).stop 2024-09-17T03:15:31.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.299+0000 7fcdf3320640 1 -- 172.21.15.87:0/2857855116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdec071c80 msgr2=0x7fcdec1e8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:31.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.299+0000 7fcdf3320640 1 --2- 172.21.15.87:0/2857855116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdec071c80 0x7fcdec1e8440 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00ea20 tx=0x7fcddc00eee0 comp rx=0 tx=0).stop 2024-09-17T03:15:31.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.299+0000 7fcdf3320640 1 -- 172.21.15.87:0/2857855116 shutdown_connections 2024-09-17T03:15:31.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.299+0000 7fcdf3320640 1 --2- 172.21.15.87:0/2857855116 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcdd8078b90 0x7fcdd807b0a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:31.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.299+0000 7fcdf3320640 1 --2- 172.21.15.87:0/2857855116 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcdec162fd0 0x7fcdec160fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:31.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.299+0000 7fcdf3320640 1 --2- 172.21.15.87:0/2857855116 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcdec15fa20 0x7fcdec139930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:31.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.299+0000 7fcdf3320640 1 --2- 172.21.15.87:0/2857855116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcdec071c80 0x7fcdec1e8440 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:31.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.299+0000 7fcdf3320640 1 -- 172.21.15.87:0/2857855116 >> 172.21.15.87:0/2857855116 conn(0x7fcdec0cdbc0 msgr2=0x7fcdec073d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:31.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.299+0000 7fcdf3320640 1 -- 172.21.15.87:0/2857855116 shutdown_connections 2024-09-17T03:15:31.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.299+0000 7fcdf3320640 1 -- 172.21.15.87:0/2857855116 wait complete. 2024-09-17T03:15:31.310 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2024-09-17T03:15:31.425 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.425+0000 7fe6d8e84640 1 -- 172.21.15.87:0/2213802683 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6d40d15d0 msgr2=0x7fe6d40d2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:31.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.425+0000 7fe6d8e84640 1 --2- 172.21.15.87:0/2213802683 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6d40d15d0 0x7fe6d40d2320 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8005fd0 tx=0x7fe6c802a540 comp rx=0 tx=0).stop 2024-09-17T03:15:31.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.425+0000 7fe6d8e84640 1 -- 172.21.15.87:0/2213802683 shutdown_connections 2024-09-17T03:15:31.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.425+0000 7fe6d8e84640 1 --2- 172.21.15.87:0/2213802683 >> v2:172.21.15.149:3300/0 conn(0x7fe6d4161800 0x7fe6d40d2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:31.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.425+0000 7fe6d8e84640 1 --2- 172.21.15.87:0/2213802683 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6d40d15d0 0x7fe6d40d2320 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:31.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.425+0000 7fe6d8e84640 1 -- 172.21.15.87:0/2213802683 >> 172.21.15.87:0/2213802683 conn(0x7fe6d409d700 msgr2=0x7fe6d40d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:31.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.425+0000 7fe6d8e84640 1 -- 172.21.15.87:0/2213802683 shutdown_connections 2024-09-17T03:15:31.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.425+0000 7fe6d8e84640 1 -- 172.21.15.87:0/2213802683 wait complete. 2024-09-17T03:15:31.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.427+0000 7fe6d8e84640 1 Processor -- start 2024-09-17T03:15:31.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.427+0000 7fe6d8e84640 1 -- start start 2024-09-17T03:15:31.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.428+0000 7fe6d8e84640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6d40b33d0 0x7fe6d415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:31.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.428+0000 7fe6d8e84640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6d4161800 0x7fe6d415bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:31.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.428+0000 7fe6d8e84640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6d40b8b30 0x7fe6d40b8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:31.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.428+0000 7fe6d8e84640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d40d3130 con 0x7fe6d40b33d0 2024-09-17T03:15:31.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.428+0000 7fe6d8e84640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d40d2fb0 con 0x7fe6d40b8b30 2024-09-17T03:15:31.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.428+0000 7fe6d8e84640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe6d40d32b0 con 0x7fe6d4161800 2024-09-17T03:15:31.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.429+0000 7fe6d259c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6d40b33d0 0x7fe6d415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:31.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.429+0000 7fe6d259c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6d40b33d0 0x7fe6d415b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53592/0 (socket says 172.21.15.87:53592) 2024-09-17T03:15:31.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.429+0000 7fe6d259c640 1 -- 172.21.15.87:0/2165502724 learned_addr learned my addr 172.21.15.87:0/2165502724 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:31.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.429+0000 7fe6d1d9b640 1 --2- 172.21.15.87:0/2165502724 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6d4161800 0x7fe6d415bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:31.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.429+0000 7fe6d2d9d640 1 --2- 172.21.15.87:0/2165502724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6d40b8b30 0x7fe6d40b8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:31.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.429+0000 7fe6d259c640 1 -- 172.21.15.87:0/2165502724 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6d4161800 msgr2=0x7fe6d415bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:31.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.429+0000 7fe6d259c640 1 --2- 172.21.15.87:0/2165502724 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6d4161800 0x7fe6d415bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:31.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.429+0000 7fe6d259c640 1 -- 172.21.15.87:0/2165502724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6d40b8b30 msgr2=0x7fe6d40b8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:31.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.429+0000 7fe6d259c640 1 --2- 172.21.15.87:0/2165502724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6d40b8b30 0x7fe6d40b8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:31.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.429+0000 7fe6d259c640 1 -- 172.21.15.87:0/2165502724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d41f90f0 con 0x7fe6d40b33d0 2024-09-17T03:15:31.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.429+0000 7fe6d2d9d640 1 --2- 172.21.15.87:0/2165502724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6d40b8b30 0x7fe6d40b8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:31.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.429+0000 7fe6d259c640 1 --2- 172.21.15.87:0/2165502724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6d40b33d0 0x7fe6d415b1f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400c730 tx=0x7fe6c400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:31.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.429+0000 7fe6c2ffd640 1 -- 172.21.15.87:0/2165502724 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4003b70 con 0x7fe6d40b33d0 2024-09-17T03:15:31.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.429+0000 7fe6c2ffd640 1 -- 172.21.15.87:0/2165502724 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6c4003d10 con 0x7fe6d40b33d0 2024-09-17T03:15:31.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.429+0000 7fe6c2ffd640 1 -- 172.21.15.87:0/2165502724 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c400aa50 con 0x7fe6d40b33d0 2024-09-17T03:15:31.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.429+0000 7fe6d8e84640 1 -- 172.21.15.87:0/2165502724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d41f93d0 con 0x7fe6d40b33d0 2024-09-17T03:15:31.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.429+0000 7fe6d8e84640 1 -- 172.21.15.87:0/2165502724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d41f9990 con 0x7fe6d40b33d0 2024-09-17T03:15:31.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.430+0000 7fe6d8e84640 1 -- 172.21.15.87:0/2165502724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d40d2320 con 0x7fe6d40b33d0 2024-09-17T03:15:31.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.431+0000 7fe6c2ffd640 1 -- 172.21.15.87:0/2165502724 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe6c400abf0 con 0x7fe6d40b33d0 2024-09-17T03:15:31.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.432+0000 7fe6c2ffd640 1 --2- 172.21.15.87:0/2165502724 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe6b4078b90 0x7fe6b407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:31.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.432+0000 7fe6c2ffd640 1 -- 172.21.15.87:0/2165502724 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7fe6c409a750 con 0x7fe6d40b33d0 2024-09-17T03:15:31.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.432+0000 7fe6d1d9b640 1 --2- 172.21.15.87:0/2165502724 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe6b4078b90 0x7fe6b407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:31.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.433+0000 7fe6d1d9b640 1 --2- 172.21.15.87:0/2165502724 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe6b4078b90 0x7fe6b407b0a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8033000 tx=0x7fe6c8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:31.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.436+0000 7fe6c2ffd640 1 -- 172.21.15.87:0/2165502724 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe6c4062e90 con 0x7fe6d40b33d0 2024-09-17T03:15:31.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:31.676+0000 7fe6d8e84640 1 -- 172.21.15.87:0/2165502724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) -- 0x7fe6d40b1300 con 0x7fe6d40b33d0 2024-09-17T03:15:32.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.679+0000 7fe6c2ffd640 1 -- 172.21.15.87:0/2165502724 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v92) ==== 137+0+0 (secure 0 0 0) 0x7fe6c4062790 con 0x7fe6d40b33d0 2024-09-17T03:15:32.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.683+0000 7fe6d8e84640 1 -- 172.21.15.87:0/2165502724 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe6b4078b90 msgr2=0x7fe6b407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:32.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.683+0000 7fe6d8e84640 1 --2- 172.21.15.87:0/2165502724 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe6b4078b90 0x7fe6b407b0a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8033000 tx=0x7fe6c8004730 comp rx=0 tx=0).stop 2024-09-17T03:15:32.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.683+0000 7fe6d8e84640 1 -- 172.21.15.87:0/2165502724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6d40b33d0 msgr2=0x7fe6d415b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:32.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.683+0000 7fe6d8e84640 1 --2- 172.21.15.87:0/2165502724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6d40b33d0 0x7fe6d415b1f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe6c400c730 tx=0x7fe6c400cbf0 comp rx=0 tx=0).stop 2024-09-17T03:15:32.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.683+0000 7fe6d8e84640 1 -- 172.21.15.87:0/2165502724 shutdown_connections 2024-09-17T03:15:32.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.683+0000 7fe6d8e84640 1 --2- 172.21.15.87:0/2165502724 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe6b4078b90 0x7fe6b407b0a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:32.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.683+0000 7fe6d8e84640 1 --2- 172.21.15.87:0/2165502724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe6d40b8b30 0x7fe6d40b8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:32.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.683+0000 7fe6d8e84640 1 --2- 172.21.15.87:0/2165502724 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe6d4161800 0x7fe6d415bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:32.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.683+0000 7fe6d8e84640 1 --2- 172.21.15.87:0/2165502724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6d40b33d0 0x7fe6d415b1f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:32.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.683+0000 7fe6d8e84640 1 -- 172.21.15.87:0/2165502724 >> 172.21.15.87:0/2165502724 conn(0x7fe6d409d700 msgr2=0x7fe6d41eac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:32.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.683+0000 7fe6d8e84640 1 -- 172.21.15.87:0/2165502724 shutdown_connections 2024-09-17T03:15:32.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.683+0000 7fe6d8e84640 1 -- 172.21.15.87:0/2165502724 wait complete. 2024-09-17T03:15:32.696 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-17T03:15:32.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.814+0000 7fc35e65c640 1 -- 172.21.15.87:0/3968045700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3580b7f10 msgr2=0x7fc35815c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:32.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.814+0000 7fc35e65c640 1 --2- 172.21.15.87:0/3968045700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3580b7f10 0x7fc35815c5f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc348009900 tx=0x7fc34802a580 comp rx=0 tx=0).stop 2024-09-17T03:15:32.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.814+0000 7fc35e65c640 1 -- 172.21.15.87:0/3968045700 shutdown_connections 2024-09-17T03:15:32.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.814+0000 7fc35e65c640 1 --2- 172.21.15.87:0/3968045700 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc3580d0230 0x7fc3580d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:32.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.814+0000 7fc35e65c640 1 --2- 172.21.15.87:0/3968045700 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3580b7f10 0x7fc35815c5f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:32.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.814+0000 7fc35e65c640 1 -- 172.21.15.87:0/3968045700 >> 172.21.15.87:0/3968045700 conn(0x7fc358069330 msgr2=0x7fc3580ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:32.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.814+0000 7fc35e65c640 1 -- 172.21.15.87:0/3968045700 shutdown_connections 2024-09-17T03:15:32.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.814+0000 7fc35e65c640 1 -- 172.21.15.87:0/3968045700 wait complete. 2024-09-17T03:15:32.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.816+0000 7fc35e65c640 1 Processor -- start 2024-09-17T03:15:32.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.816+0000 7fc35e65c640 1 -- start start 2024-09-17T03:15:32.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.818+0000 7fc35e65c640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc3580b3d60 0x7fc3581f5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:32.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.818+0000 7fc35e65c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3580b7f10 0x7fc3581f8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:32.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.818+0000 7fc35e65c640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3580d0230 0x7fc3581f9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:32.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.818+0000 7fc35e65c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc3580d0b80 con 0x7fc3580b7f10 2024-09-17T03:15:32.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.818+0000 7fc35e65c640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc3580b7d20 con 0x7fc3580d0230 2024-09-17T03:15:32.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.818+0000 7fc35e65c640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc3580d0d00 con 0x7fc3580b3d60 2024-09-17T03:15:32.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.818+0000 7fc3577fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3580b7f10 0x7fc3581f8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:32.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.818+0000 7fc3577fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3580b7f10 0x7fc3581f8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53600/0 (socket says 172.21.15.87:53600) 2024-09-17T03:15:32.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.818+0000 7fc3577fe640 1 -- 172.21.15.87:0/1263307905 learned_addr learned my addr 172.21.15.87:0/1263307905 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:32.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.818+0000 7fc35cbf9640 1 --2- 172.21.15.87:0/1263307905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3580d0230 0x7fc3581f9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:32.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.818+0000 7fc357fff640 1 --2- 172.21.15.87:0/1263307905 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc3580b3d60 0x7fc3581f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:32.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.818+0000 7fc3577fe640 1 -- 172.21.15.87:0/1263307905 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc3580b3d60 msgr2=0x7fc3581f5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:32.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.818+0000 7fc3577fe640 1 --2- 172.21.15.87:0/1263307905 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc3580b3d60 0x7fc3581f5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:32.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.818+0000 7fc3577fe640 1 -- 172.21.15.87:0/1263307905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3580d0230 msgr2=0x7fc3581f9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:32.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.818+0000 7fc3577fe640 1 --2- 172.21.15.87:0/1263307905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3580d0230 0x7fc3581f9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:32.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.818+0000 7fc3577fe640 1 -- 172.21.15.87:0/1263307905 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3581f6580 con 0x7fc3580b7f10 2024-09-17T03:15:32.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.818+0000 7fc3577fe640 1 --2- 172.21.15.87:0/1263307905 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3580b7f10 0x7fc3581f8f20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00c3a0 tx=0x7fc34c00c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:32.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.819+0000 7fc354ff9640 1 -- 172.21.15.87:0/1263307905 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c017070 con 0x7fc3580b7f10 2024-09-17T03:15:32.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.819+0000 7fc354ff9640 1 -- 172.21.15.87:0/1263307905 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc34c0053c0 con 0x7fc3580b7f10 2024-09-17T03:15:32.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.819+0000 7fc35e65c640 1 -- 172.21.15.87:0/1263307905 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3581f6860 con 0x7fc3580b7f10 2024-09-17T03:15:32.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.819+0000 7fc354ff9640 1 -- 172.21.15.87:0/1263307905 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34c003850 con 0x7fc3580b7f10 2024-09-17T03:15:32.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.819+0000 7fc35e65c640 1 -- 172.21.15.87:0/1263307905 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc35806e270 con 0x7fc3580b7f10 2024-09-17T03:15:32.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.820+0000 7fc35e65c640 1 -- 172.21.15.87:0/1263307905 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3580d11d0 con 0x7fc3580b7f10 2024-09-17T03:15:32.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.820+0000 7fc354ff9640 1 -- 172.21.15.87:0/1263307905 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc34c01a070 con 0x7fc3580b7f10 2024-09-17T03:15:32.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.821+0000 7fc354ff9640 1 --2- 172.21.15.87:0/1263307905 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc344078b90 0x7fc34407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:32.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.821+0000 7fc354ff9640 1 -- 172.21.15.87:0/1263307905 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7fc34c099cf0 con 0x7fc3580b7f10 2024-09-17T03:15:32.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.821+0000 7fc357fff640 1 --2- 172.21.15.87:0/1263307905 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc344078b90 0x7fc34407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:32.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.822+0000 7fc357fff640 1 --2- 172.21.15.87:0/1263307905 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc344078b90 0x7fc34407b0a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc348033000 tx=0x7fc348004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:32.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:32.825+0000 7fc354ff9640 1 -- 172.21.15.87:0/1263307905 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc34c09e070 con 0x7fc3580b7f10 2024-09-17T03:15:33.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.000+0000 7fc35e65c640 1 -- 172.21.15.87:0/1263307905 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fc358076410 con 0x7fc3580b7f10 2024-09-17T03:15:33.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.001+0000 7fc354ff9640 1 -- 172.21.15.87:0/1263307905 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+2294 (secure 0 0 0) 0x7fc34c061d60 con 0x7fc3580b7f10 2024-09-17T03:15:33.001 INFO:teuthology.orchestra.run.smithi087.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-17T03:15:33.001 INFO:teuthology.orchestra.run.smithi087.stdout:mon advanced mon_op_complaint_time 120 2024-09-17T03:15:33.001 INFO:teuthology.orchestra.run.smithi087.stdout:mgr advanced client_mount_timeout 30 2024-09-17T03:15:33.001 INFO:teuthology.orchestra.run.smithi087.stdout:mgr advanced debug_client 20/20 2024-09-17T03:15:33.001 INFO:teuthology.orchestra.run.smithi087.stdout:mgr advanced debug_mgr 20/20 2024-09-17T03:15:33.001 INFO:teuthology.orchestra.run.smithi087.stdout:mgr advanced debug_ms 1/1 2024-09-17T03:15:33.001 INFO:teuthology.orchestra.run.smithi087.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-17T03:15:33.001 INFO:teuthology.orchestra.run.smithi087.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-17T03:15:33.001 INFO:teuthology.orchestra.run.smithi087.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 4490.708381 2024-09-17T03:15:33.002 INFO:teuthology.orchestra.run.smithi087.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4822.941320 2024-09-17T03:15:33.002 INFO:teuthology.orchestra.run.smithi087.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 5249.365509 2024-09-17T03:15:33.002 INFO:teuthology.orchestra.run.smithi087.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4538.303788 2024-09-17T03:15:33.002 INFO:teuthology.orchestra.run.smithi087.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 6974.722182 2024-09-17T03:15:33.002 INFO:teuthology.orchestra.run.smithi087.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 7605.766974 2024-09-17T03:15:33.002 INFO:teuthology.orchestra.run.smithi087.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 5406.650094 2024-09-17T03:15:33.002 INFO:teuthology.orchestra.run.smithi087.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5658.926090 2024-09-17T03:15:33.002 INFO:teuthology.orchestra.run.smithi087.stdout:mds advanced debug_mds 20/20 2024-09-17T03:15:33.002 INFO:teuthology.orchestra.run.smithi087.stdout:mds advanced debug_mds_balancer 20/20 2024-09-17T03:15:33.002 INFO:teuthology.orchestra.run.smithi087.stdout:mds advanced debug_ms 2/2 2024-09-17T03:15:33.002 INFO:teuthology.orchestra.run.smithi087.stdout:mds dev mds_debug_frag true 2024-09-17T03:15:33.002 INFO:teuthology.orchestra.run.smithi087.stdout:mds dev mds_debug_scatterstat true 2024-09-17T03:15:33.002 INFO:teuthology.orchestra.run.smithi087.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-17T03:15:33.002 INFO:teuthology.orchestra.run.smithi087.stdout:mds dev mds_verify_scatter true 2024-09-17T03:15:33.003 INFO:teuthology.orchestra.run.smithi087.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-17T03:15:33.003 INFO:teuthology.orchestra.run.smithi087.stdout:mds advanced rados_mon_op_timeout 900 2024-09-17T03:15:33.003 INFO:teuthology.orchestra.run.smithi087.stdout:mds advanced rados_osd_op_timeout 900 2024-09-17T03:15:33.003 INFO:teuthology.orchestra.run.smithi087.stdout:client advanced client_mount_timeout 600 2024-09-17T03:15:33.003 INFO:teuthology.orchestra.run.smithi087.stdout:client advanced debug_client 20/20 2024-09-17T03:15:33.003 INFO:teuthology.orchestra.run.smithi087.stdout:client advanced debug_ms 1/1 2024-09-17T03:15:33.003 INFO:teuthology.orchestra.run.smithi087.stdout:client advanced rados_mon_op_timeout 900 2024-09-17T03:15:33.003 INFO:teuthology.orchestra.run.smithi087.stdout:client advanced rados_osd_op_timeout 900 2024-09-17T03:15:33.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.003+0000 7fc35e65c640 1 -- 172.21.15.87:0/1263307905 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc344078b90 msgr2=0x7fc34407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:33.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.003+0000 7fc35e65c640 1 --2- 172.21.15.87:0/1263307905 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc344078b90 0x7fc34407b0a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc348033000 tx=0x7fc348004730 comp rx=0 tx=0).stop 2024-09-17T03:15:33.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.003+0000 7fc35e65c640 1 -- 172.21.15.87:0/1263307905 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3580b7f10 msgr2=0x7fc3581f8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:33.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.003+0000 7fc35e65c640 1 --2- 172.21.15.87:0/1263307905 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3580b7f10 0x7fc3581f8f20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc34c00c3a0 tx=0x7fc34c00c7a0 comp rx=0 tx=0).stop 2024-09-17T03:15:33.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.003+0000 7fc35e65c640 1 -- 172.21.15.87:0/1263307905 shutdown_connections 2024-09-17T03:15:33.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.003+0000 7fc35e65c640 1 --2- 172.21.15.87:0/1263307905 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc344078b90 0x7fc34407b0a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:33.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.003+0000 7fc35e65c640 1 --2- 172.21.15.87:0/1263307905 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3580d0230 0x7fc3581f9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:33.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.003+0000 7fc35e65c640 1 --2- 172.21.15.87:0/1263307905 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3580b7f10 0x7fc3581f8f20 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:33.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.003+0000 7fc35e65c640 1 --2- 172.21.15.87:0/1263307905 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc3580b3d60 0x7fc3581f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:33.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.003+0000 7fc35e65c640 1 -- 172.21.15.87:0/1263307905 >> 172.21.15.87:0/1263307905 conn(0x7fc358069330 msgr2=0x7fc358075c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:33.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.003+0000 7fc35e65c640 1 -- 172.21.15.87:0/1263307905 shutdown_connections 2024-09-17T03:15:33.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.003+0000 7fc35e65c640 1 -- 172.21.15.87:0/1263307905 wait complete. 2024-09-17T03:15:33.014 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-17T03:15:33.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.129+0000 7fa6bd539640 1 -- 172.21.15.87:0/768735564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6b80b3080 msgr2=0x7fa6b80cfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:33.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.129+0000 7fa6bd539640 1 --2- 172.21.15.87:0/768735564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6b80b3080 0x7fa6b80cfe60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8005fd0 tx=0x7fa6a802a5d0 comp rx=0 tx=0).stop 2024-09-17T03:15:33.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.129+0000 7fa6bd539640 1 -- 172.21.15.87:0/768735564 shutdown_connections 2024-09-17T03:15:33.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.129+0000 7fa6bd539640 1 --2- 172.21.15.87:0/768735564 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa6b80b3e00 0x7fa6b80cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:33.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.129+0000 7fa6bd539640 1 --2- 172.21.15.87:0/768735564 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6b80b3080 0x7fa6b80cfe60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:33.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.129+0000 7fa6bd539640 1 -- 172.21.15.87:0/768735564 >> 172.21.15.87:0/768735564 conn(0x7fa6b808bfc0 msgr2=0x7fa6b809d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:33.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.129+0000 7fa6bd539640 1 -- 172.21.15.87:0/768735564 shutdown_connections 2024-09-17T03:15:33.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.129+0000 7fa6bd539640 1 -- 172.21.15.87:0/768735564 wait complete. 2024-09-17T03:15:33.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.130+0000 7fa6bd539640 1 Processor -- start 2024-09-17T03:15:33.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.131+0000 7fa6bd539640 1 -- start start 2024-09-17T03:15:33.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.132+0000 7fa6bd539640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6b80b3080 0x7fa6b80cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:33.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.132+0000 7fa6bd539640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6b80b3e00 0x7fa6b815ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:33.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.132+0000 7fa6bd539640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa6b80d4350 0x7fa6b80cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:33.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.132+0000 7fa6bd539640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b80d2a20 con 0x7fa6b80b3e00 2024-09-17T03:15:33.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.132+0000 7fa6bd539640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b80d28a0 con 0x7fa6b80b3080 2024-09-17T03:15:33.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.132+0000 7fa6bd539640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa6b80d2ba0 con 0x7fa6b80d4350 2024-09-17T03:15:33.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.132+0000 7fa6b67fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6b80b3e00 0x7fa6b815ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:33.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.132+0000 7fa6b67fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6b80b3e00 0x7fa6b815ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53612/0 (socket says 172.21.15.87:53612) 2024-09-17T03:15:33.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.132+0000 7fa6b67fc640 1 -- 172.21.15.87:0/2592747301 learned_addr learned my addr 172.21.15.87:0/2592747301 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:33.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.132+0000 7fa6b6ffd640 1 --2- 172.21.15.87:0/2592747301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6b80b3080 0x7fa6b80cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:33.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.132+0000 7fa6b77fe640 1 --2- 172.21.15.87:0/2592747301 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa6b80d4350 0x7fa6b80cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:33.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.132+0000 7fa6b67fc640 1 -- 172.21.15.87:0/2592747301 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa6b80d4350 msgr2=0x7fa6b80cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:33.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.132+0000 7fa6b67fc640 1 --2- 172.21.15.87:0/2592747301 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa6b80d4350 0x7fa6b80cd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:33.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.132+0000 7fa6b67fc640 1 -- 172.21.15.87:0/2592747301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6b80b3080 msgr2=0x7fa6b80cb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:33.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.132+0000 7fa6b67fc640 1 --2- 172.21.15.87:0/2592747301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6b80b3080 0x7fa6b80cb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:33.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.132+0000 7fa6b67fc640 1 -- 172.21.15.87:0/2592747301 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b80cea80 con 0x7fa6b80b3e00 2024-09-17T03:15:33.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.132+0000 7fa6b6ffd640 1 --2- 172.21.15.87:0/2592747301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6b80b3080 0x7fa6b80cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:15:33.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.132+0000 7fa6b67fc640 1 --2- 172.21.15.87:0/2592747301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6b80b3e00 0x7fa6b815ffe0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00e7f0 tx=0x7fa6ac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:33.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.133+0000 7fa69f7fe640 1 -- 172.21.15.87:0/2592747301 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac00eea0 con 0x7fa6b80b3e00 2024-09-17T03:15:33.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.133+0000 7fa69f7fe640 1 -- 172.21.15.87:0/2592747301 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6ac00cdb0 con 0x7fa6b80b3e00 2024-09-17T03:15:33.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.133+0000 7fa69f7fe640 1 -- 172.21.15.87:0/2592747301 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac010690 con 0x7fa6b80b3e00 2024-09-17T03:15:33.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.133+0000 7fa6bd539640 1 -- 172.21.15.87:0/2592747301 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b80ce4c0 con 0x7fa6b80b3e00 2024-09-17T03:15:33.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.133+0000 7fa6bd539640 1 -- 172.21.15.87:0/2592747301 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b815dcd0 con 0x7fa6b80b3e00 2024-09-17T03:15:33.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.134+0000 7fa6bd539640 1 -- 172.21.15.87:0/2592747301 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b80d3070 con 0x7fa6b80b3e00 2024-09-17T03:15:33.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.134+0000 7fa69f7fe640 1 -- 172.21.15.87:0/2592747301 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa6ac010830 con 0x7fa6b80b3e00 2024-09-17T03:15:33.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.135+0000 7fa69f7fe640 1 --2- 172.21.15.87:0/2592747301 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa6a4078bc0 0x7fa6a407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:33.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.135+0000 7fa69f7fe640 1 -- 172.21.15.87:0/2592747301 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7fa6ac09b3e0 con 0x7fa6b80b3e00 2024-09-17T03:15:33.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.135+0000 7fa6b6ffd640 1 --2- 172.21.15.87:0/2592747301 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa6a4078bc0 0x7fa6a407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:33.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.136+0000 7fa6b6ffd640 1 --2- 172.21.15.87:0/2592747301 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa6a4078bc0 0x7fa6a407b0d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8033000 tx=0x7fa6a8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:33.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.139+0000 7fa69f7fe640 1 -- 172.21.15.87:0/2592747301 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fa6ac063b20 con 0x7fa6b80b3e00 2024-09-17T03:15:33.314 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.314+0000 7fa6bd539640 1 -- 172.21.15.87:0/2592747301 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa6b80ce250 con 0x7fa6b80b3e00 2024-09-17T03:15:33.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.318+0000 7fa69f7fe640 1 -- 172.21.15.87:0/2592747301 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7fa6ac063420 con 0x7fa6b80b3e00 2024-09-17T03:15:33.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.320+0000 7fa6bd539640 1 -- 172.21.15.87:0/2592747301 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa6a4078bc0 msgr2=0x7fa6a407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:33.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.320+0000 7fa6bd539640 1 --2- 172.21.15.87:0/2592747301 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa6a4078bc0 0x7fa6a407b0d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8033000 tx=0x7fa6a8004730 comp rx=0 tx=0).stop 2024-09-17T03:15:33.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.320+0000 7fa6bd539640 1 -- 172.21.15.87:0/2592747301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6b80b3e00 msgr2=0x7fa6b815ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:33.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.320+0000 7fa6bd539640 1 --2- 172.21.15.87:0/2592747301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6b80b3e00 0x7fa6b815ffe0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00e7f0 tx=0x7fa6ac00ecb0 comp rx=0 tx=0).stop 2024-09-17T03:15:33.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.320+0000 7fa6bd539640 1 -- 172.21.15.87:0/2592747301 shutdown_connections 2024-09-17T03:15:33.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.320+0000 7fa6bd539640 1 --2- 172.21.15.87:0/2592747301 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa6a4078bc0 0x7fa6a407b0d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:33.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.320+0000 7fa6bd539640 1 --2- 172.21.15.87:0/2592747301 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa6b80d4350 0x7fa6b80cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:33.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.320+0000 7fa6bd539640 1 --2- 172.21.15.87:0/2592747301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6b80b3e00 0x7fa6b815ffe0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:33.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.320+0000 7fa6bd539640 1 --2- 172.21.15.87:0/2592747301 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa6b80b3080 0x7fa6b80cb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:33.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.320+0000 7fa6bd539640 1 -- 172.21.15.87:0/2592747301 >> 172.21.15.87:0/2592747301 conn(0x7fa6b808bfc0 msgr2=0x7fa6b80b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:33.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.321+0000 7fa6bd539640 1 -- 172.21.15.87:0/2592747301 shutdown_connections 2024-09-17T03:15:33.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.321+0000 7fa6bd539640 1 -- 172.21.15.87:0/2592747301 wait complete. 2024-09-17T03:15:33.331 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T03:15:33.332 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:15:33.391 DEBUG:teuthology.orchestra.run.smithi149:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:15:33.440 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T03:15:33.440 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:15:33.441 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:15:33.456 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=156 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:15:33.456 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:15:33.457 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:15:33.474 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=1994716,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=3199368k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20815 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=1599680k,nr_inodes=399920,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=3199368k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:15:33.474 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:15:33.474 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:15:33.511 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=156 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:15:33.512 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:15:33.512 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:15:33.531 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=1994716,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=3199368k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20815 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=1599680k,nr_inodes=399920,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=3199368k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:15:33.531 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-17T03:15:33.531 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:15:33.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.664+0000 7f3e574d3640 1 -- 172.21.15.87:0/487122409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e5015f3d0 msgr2=0x7f3e5015f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:33.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.664+0000 7f3e574d3640 1 --2- 172.21.15.87:0/487122409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e5015f3d0 0x7f3e5015f7e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3e48005fd0 tx=0x7f3e4802a9f0 comp rx=0 tx=0).stop 2024-09-17T03:15:33.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.664+0000 7f3e574d3640 1 -- 172.21.15.87:0/487122409 shutdown_connections 2024-09-17T03:15:33.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.664+0000 7f3e574d3640 1 --2- 172.21.15.87:0/487122409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e5015f3d0 0x7f3e5015f7e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:33.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.664+0000 7f3e574d3640 1 --2- 172.21.15.87:0/487122409 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3e500c9770 0x7f3e500c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:33.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.664+0000 7f3e574d3640 1 -- 172.21.15.87:0/487122409 >> 172.21.15.87:0/487122409 conn(0x7f3e500b20f0 msgr2=0x7f3e500b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:33.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.664+0000 7f3e574d3640 1 -- 172.21.15.87:0/487122409 shutdown_connections 2024-09-17T03:15:33.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.664+0000 7f3e574d3640 1 -- 172.21.15.87:0/487122409 wait complete. 2024-09-17T03:15:33.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.666+0000 7f3e574d3640 1 Processor -- start 2024-09-17T03:15:33.665 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.666+0000 7f3e574d3640 1 -- start start 2024-09-17T03:15:33.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.667+0000 7f3e574d3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e500c9770 0x7f3e5015c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:33.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.667+0000 7f3e574d3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e500cff20 0x7f3e50145e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:33.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.667+0000 7f3e574d3640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3e5015f3d0 0x7f3e500b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:33.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.667+0000 7f3e574d3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3e50147aa0 con 0x7f3e500c9770 2024-09-17T03:15:33.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.667+0000 7f3e574d3640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3e50147920 con 0x7f3e500cff20 2024-09-17T03:15:33.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.667+0000 7f3e574d3640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3e50147c20 con 0x7f3e5015f3d0 2024-09-17T03:15:33.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.667+0000 7f3e5526f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e500c9770 0x7f3e5015c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:33.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.667+0000 7f3e5526f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e500c9770 0x7f3e5015c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53624/0 (socket says 172.21.15.87:53624) 2024-09-17T03:15:33.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.667+0000 7f3e5526f640 1 -- 172.21.15.87:0/2100231641 learned_addr learned my addr 172.21.15.87:0/2100231641 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:33.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.667+0000 7f3e55a70640 1 --2- 172.21.15.87:0/2100231641 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3e5015f3d0 0x7f3e500b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:33.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.667+0000 7f3e54a6e640 1 --2- 172.21.15.87:0/2100231641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e500cff20 0x7f3e50145e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:33.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.667+0000 7f3e5526f640 1 -- 172.21.15.87:0/2100231641 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3e5015f3d0 msgr2=0x7f3e500b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:33.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.667+0000 7f3e5526f640 1 --2- 172.21.15.87:0/2100231641 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3e5015f3d0 0x7f3e500b10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:33.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.667+0000 7f3e5526f640 1 -- 172.21.15.87:0/2100231641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e500cff20 msgr2=0x7f3e50145e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:33.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.667+0000 7f3e5526f640 1 --2- 172.21.15.87:0/2100231641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e500cff20 0x7f3e50145e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:33.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.667+0000 7f3e5526f640 1 -- 172.21.15.87:0/2100231641 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e5015c720 con 0x7f3e500c9770 2024-09-17T03:15:33.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.668+0000 7f3e5526f640 1 --2- 172.21.15.87:0/2100231641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e500c9770 0x7f3e5015c1e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00b3f0 tx=0x7f3e4c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:33.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.668+0000 7f3e45ffb640 1 -- 172.21.15.87:0/2100231641 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c015020 con 0x7f3e500c9770 2024-09-17T03:15:33.669 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.668+0000 7f3e45ffb640 1 -- 172.21.15.87:0/2100231641 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e4c011920 con 0x7f3e500c9770 2024-09-17T03:15:33.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.668+0000 7f3e574d3640 1 -- 172.21.15.87:0/2100231641 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e501f8e40 con 0x7f3e500c9770 2024-09-17T03:15:33.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.668+0000 7f3e45ffb640 1 -- 172.21.15.87:0/2100231641 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c011d70 con 0x7f3e500c9770 2024-09-17T03:15:33.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.668+0000 7f3e574d3640 1 -- 172.21.15.87:0/2100231641 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e501f9400 con 0x7f3e500c9770 2024-09-17T03:15:33.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.669+0000 7f3e574d3640 1 -- 172.21.15.87:0/2100231641 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e50148130 con 0x7f3e500c9770 2024-09-17T03:15:33.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.669+0000 7f3e45ffb640 1 -- 172.21.15.87:0/2100231641 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3e4c018020 con 0x7f3e500c9770 2024-09-17T03:15:33.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.670+0000 7f3e45ffb640 1 --2- 172.21.15.87:0/2100231641 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3e34078b90 0x7f3e3407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:33.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.670+0000 7f3e45ffb640 1 -- 172.21.15.87:0/2100231641 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7f3e4c09a530 con 0x7f3e500c9770 2024-09-17T03:15:33.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.670+0000 7f3e54a6e640 1 --2- 172.21.15.87:0/2100231641 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3e34078b90 0x7f3e3407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:33.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.671+0000 7f3e54a6e640 1 --2- 172.21.15.87:0/2100231641 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3e34078b90 0x7f3e3407b0a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3e500cdf50 tx=0x7f3e40005d80 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:33.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.675+0000 7f3e45ffb640 1 -- 172.21.15.87:0/2100231641 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f3e4c01f030 con 0x7f3e500c9770 2024-09-17T03:15:33.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.887+0000 7f3e574d3640 1 -- 172.21.15.87:0/2100231641 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3e501f88d0 con 0x7f3e500c9770 2024-09-17T03:15:33.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.888+0000 7f3e45ffb640 1 -- 172.21.15.87:0/2100231641 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+3911 (secure 0 0 0) 0x7f3e4c062d90 con 0x7f3e500c9770 2024-09-17T03:15:33.888 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:33.888 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":16,"btime":"2024-09-17T03:15:26:369786+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4634,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6836/1081794135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":1081794135},{"type":"v1","addr":"172.21.15.149:6836","nonce":1081794135}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":4637,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/2152665509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6835","nonce":2152665509},{"type":"v1","addr":"172.21.15.149:6837","nonce":2152665509}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5001,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/704821751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":704821751},{"type":"v1","addr":"172.21.15.87:6835","nonce":704821751}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":5391,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/999747280","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":999747280},{"type":"v1","addr":"172.21.15.87:6834","nonce":999747280}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":16}],"filesystems":[]} 2024-09-17T03:15:33.888 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 16 2024-09-17T03:15:33.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.890+0000 7f3e574d3640 1 -- 172.21.15.87:0/2100231641 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3e34078b90 msgr2=0x7f3e3407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:33.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.890+0000 7f3e574d3640 1 --2- 172.21.15.87:0/2100231641 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3e34078b90 0x7f3e3407b0a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3e500cdf50 tx=0x7f3e40005d80 comp rx=0 tx=0).stop 2024-09-17T03:15:33.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.890+0000 7f3e574d3640 1 -- 172.21.15.87:0/2100231641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e500c9770 msgr2=0x7f3e5015c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:33.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.890+0000 7f3e574d3640 1 --2- 172.21.15.87:0/2100231641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e500c9770 0x7f3e5015c1e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3e4c00b3f0 tx=0x7f3e4c00b8b0 comp rx=0 tx=0).stop 2024-09-17T03:15:33.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.890+0000 7f3e574d3640 1 -- 172.21.15.87:0/2100231641 shutdown_connections 2024-09-17T03:15:33.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.890+0000 7f3e574d3640 1 --2- 172.21.15.87:0/2100231641 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3e34078b90 0x7f3e3407b0a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:33.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.890+0000 7f3e574d3640 1 --2- 172.21.15.87:0/2100231641 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f3e5015f3d0 0x7f3e500b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:33.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.890+0000 7f3e574d3640 1 --2- 172.21.15.87:0/2100231641 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3e500cff20 0x7f3e50145e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:33.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.890+0000 7f3e574d3640 1 --2- 172.21.15.87:0/2100231641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3e500c9770 0x7f3e5015c1e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:33.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.890+0000 7f3e574d3640 1 -- 172.21.15.87:0/2100231641 >> 172.21.15.87:0/2100231641 conn(0x7f3e500b20f0 msgr2=0x7f3e500b5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:33.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.890+0000 7f3e574d3640 1 -- 172.21.15.87:0/2100231641 shutdown_connections 2024-09-17T03:15:33.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:33.890+0000 7f3e574d3640 1 -- 172.21.15.87:0/2100231641 wait complete. 2024-09-17T03:15:33.901 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-17T03:15:33.902 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-17T03:15:33.902 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T03:15:33.902 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:15:33.902 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-17T03:15:33.902 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-17T03:15:33.902 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T03:15:33.902 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:15:33.903 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-17T03:15:33.903 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-17T03:15:33.903 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T03:15:33.903 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:15:33.903 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-17T03:15:33.903 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-17T03:15:33.903 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T03:15:33.904 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:15:33.912 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T03:15:33.912 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-17T03:15:33.915 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T03:15:33.915 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-17T03:15:33.916 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T03:15:33.916 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-17T03:15:33.917 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T03:15:33.917 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-17T03:15:33.957 INFO:tasks.ceph.mds.a:Started 2024-09-17T03:15:33.957 DEBUG:teuthology.parallel:result is None 2024-09-17T03:15:33.958 INFO:tasks.ceph.mds.c:Started 2024-09-17T03:15:33.958 DEBUG:teuthology.parallel:result is None 2024-09-17T03:15:33.959 INFO:tasks.ceph.mds.b:Started 2024-09-17T03:15:33.960 DEBUG:teuthology.parallel:result is None 2024-09-17T03:15:33.960 INFO:tasks.ceph.mds.d:Started 2024-09-17T03:15:33.960 DEBUG:teuthology.parallel:result is None 2024-09-17T03:15:33.961 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-17T03:15:34.043 INFO:tasks.ceph.mds.a.smithi087.stdout:starting mds.a at 2024-09-17T03:15:34.044 INFO:tasks.ceph.mds.c.smithi087.stdout:starting mds.c at 2024-09-17T03:15:34.055 INFO:tasks.ceph.mds.b.smithi149.stdout:starting mds.b at 2024-09-17T03:15:34.056 INFO:tasks.ceph.mds.d.smithi149.stdout:starting mds.d at 2024-09-17T03:15:34.056 INFO:tasks.ceph.mds.b.smithi149.stderr:2024-09-17T03:15:34.057+0000 7f78a04ce640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-17T03:15:34.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.080+0000 7ff79d3ed640 1 -- 172.21.15.87:0/1767129102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7980d06e0 msgr2=0x7ff798074310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:34.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.080+0000 7ff79d3ed640 1 --2- 172.21.15.87:0/1767129102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7980d06e0 0x7ff798074310 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff780009830 tx=0x7ff78002a480 comp rx=0 tx=0).stop 2024-09-17T03:15:34.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.080+0000 7ff79d3ed640 1 -- 172.21.15.87:0/1767129102 shutdown_connections 2024-09-17T03:15:34.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.080+0000 7ff79d3ed640 1 --2- 172.21.15.87:0/1767129102 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7980d06e0 0x7ff798074310 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:34.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.080+0000 7ff79d3ed640 1 --2- 172.21.15.87:0/1767129102 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff798160f30 0x7ff7980d01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:34.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.080+0000 7ff79d3ed640 1 -- 172.21.15.87:0/1767129102 >> 172.21.15.87:0/1767129102 conn(0x7ff7980b2d10 msgr2=0x7ff7980b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:34.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.080+0000 7ff79d3ed640 1 -- 172.21.15.87:0/1767129102 shutdown_connections 2024-09-17T03:15:34.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.080+0000 7ff79d3ed640 1 -- 172.21.15.87:0/1767129102 wait complete. 2024-09-17T03:15:34.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.082+0000 7ff79d3ed640 1 Processor -- start 2024-09-17T03:15:34.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.082+0000 7ff79d3ed640 1 -- start start 2024-09-17T03:15:34.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.083+0000 7ff79d3ed640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff798074850 0x7ff798155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:34.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.083+0000 7ff79d3ed640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff798160f30 0x7ff7981f7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:34.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.083+0000 7ff79d3ed640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7981f75d0 0x7ff79814f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:34.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.083+0000 7ff79d3ed640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff79815d230 con 0x7ff798160f30 2024-09-17T03:15:34.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.083+0000 7ff79d3ed640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff79815d0b0 con 0x7ff7981f75d0 2024-09-17T03:15:34.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.083+0000 7ff79d3ed640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff79815d3b0 con 0x7ff798074850 2024-09-17T03:15:34.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.083+0000 7ff78ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff798160f30 0x7ff7981f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:34.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.084+0000 7ff78ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff798160f30 0x7ff7981f7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:53660/0 (socket says 172.21.15.87:53660) 2024-09-17T03:15:34.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.084+0000 7ff78ffff640 1 -- 172.21.15.87:0/2598699532 learned_addr learned my addr 172.21.15.87:0/2598699532 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:34.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.084+0000 7ff7977fe640 1 --2- 172.21.15.87:0/2598699532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7981f75d0 0x7ff79814f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:34.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.084+0000 7ff796ffd640 1 --2- 172.21.15.87:0/2598699532 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff798074850 0x7ff798155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:34.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.084+0000 7ff78ffff640 1 -- 172.21.15.87:0/2598699532 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff798074850 msgr2=0x7ff798155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:34.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.084+0000 7ff78ffff640 1 --2- 172.21.15.87:0/2598699532 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff798074850 0x7ff798155850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:34.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.084+0000 7ff78ffff640 1 -- 172.21.15.87:0/2598699532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7981f75d0 msgr2=0x7ff79814f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:34.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.084+0000 7ff78ffff640 1 --2- 172.21.15.87:0/2598699532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7981f75d0 0x7ff79814f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:34.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.084+0000 7ff78ffff640 1 -- 172.21.15.87:0/2598699532 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7981f3570 con 0x7ff798160f30 2024-09-17T03:15:34.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.084+0000 7ff78ffff640 1 --2- 172.21.15.87:0/2598699532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff798160f30 0x7ff7981f7090 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff780000c00 tx=0x7ff780033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:34.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.084+0000 7ff78f7fe640 1 -- 172.21.15.87:0/2598699532 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff780005c90 con 0x7ff798160f30 2024-09-17T03:15:34.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.084+0000 7ff78f7fe640 1 -- 172.21.15.87:0/2598699532 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff780031e50 con 0x7ff798160f30 2024-09-17T03:15:34.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.084+0000 7ff78f7fe640 1 -- 172.21.15.87:0/2598699532 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff780033310 con 0x7ff798160f30 2024-09-17T03:15:34.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.084+0000 7ff79d3ed640 1 -- 172.21.15.87:0/2598699532 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7981f3850 con 0x7ff798160f30 2024-09-17T03:15:34.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.084+0000 7ff79d3ed640 1 -- 172.21.15.87:0/2598699532 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff798158360 con 0x7ff798160f30 2024-09-17T03:15:34.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.085+0000 7ff79d3ed640 1 -- 172.21.15.87:0/2598699532 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7981561a0 con 0x7ff798160f30 2024-09-17T03:15:34.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.086+0000 7ff78f7fe640 1 -- 172.21.15.87:0/2598699532 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff7800334b0 con 0x7ff798160f30 2024-09-17T03:15:34.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.086+0000 7ff78f7fe640 1 --2- 172.21.15.87:0/2598699532 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff778078bc0 0x7ff77807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:34.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.087+0000 7ff78f7fe640 1 -- 172.21.15.87:0/2598699532 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4943+0+0 (secure 0 0 0) 0x7ff7800bbd80 con 0x7ff798160f30 2024-09-17T03:15:34.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.087+0000 7ff796ffd640 1 --2- 172.21.15.87:0/2598699532 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff778078bc0 0x7ff77807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:34.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.087+0000 7ff796ffd640 1 --2- 172.21.15.87:0/2598699532 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff778078bc0 0x7ff77807b0d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff79815dea0 tx=0x7ff788005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:34.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.091+0000 7ff78f7fe640 1 -- 172.21.15.87:0/2598699532 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff78003f030 con 0x7ff798160f30 2024-09-17T03:15:34.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:34.277+0000 7ff79d3ed640 1 -- 172.21.15.87:0/2598699532 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff79815d8d0 con 0x7ff798160f30 2024-09-17T03:15:35.330 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.330+0000 7ff78f7fe640 1 -- 172.21.15.87:0/2598699532 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v46) ==== 97+0+0 (secure 0 0 0) 0x7ff7800844d0 con 0x7ff798160f30 2024-09-17T03:15:35.330 INFO:teuthology.orchestra.run.smithi087.stderr: removed all blocklist entries 2024-09-17T03:15:35.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.333+0000 7ff79d3ed640 1 -- 172.21.15.87:0/2598699532 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff778078bc0 msgr2=0x7ff77807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:35.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.333+0000 7ff79d3ed640 1 --2- 172.21.15.87:0/2598699532 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff778078bc0 0x7ff77807b0d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff79815dea0 tx=0x7ff788005f90 comp rx=0 tx=0).stop 2024-09-17T03:15:35.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.333+0000 7ff79d3ed640 1 -- 172.21.15.87:0/2598699532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff798160f30 msgr2=0x7ff7981f7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:35.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.333+0000 7ff79d3ed640 1 --2- 172.21.15.87:0/2598699532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff798160f30 0x7ff7981f7090 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff780000c00 tx=0x7ff780033c20 comp rx=0 tx=0).stop 2024-09-17T03:15:35.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.333+0000 7ff79d3ed640 1 -- 172.21.15.87:0/2598699532 shutdown_connections 2024-09-17T03:15:35.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.333+0000 7ff79d3ed640 1 --2- 172.21.15.87:0/2598699532 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff778078bc0 0x7ff77807b0d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:35.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.333+0000 7ff79d3ed640 1 --2- 172.21.15.87:0/2598699532 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff7981f75d0 0x7ff79814f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:35.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.333+0000 7ff79d3ed640 1 --2- 172.21.15.87:0/2598699532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff798160f30 0x7ff7981f7090 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:35.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.333+0000 7ff79d3ed640 1 --2- 172.21.15.87:0/2598699532 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff798074850 0x7ff798155850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:35.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.333+0000 7ff79d3ed640 1 -- 172.21.15.87:0/2598699532 >> 172.21.15.87:0/2598699532 conn(0x7ff7980b2d10 msgr2=0x7ff79815e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:35.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.333+0000 7ff79d3ed640 1 -- 172.21.15.87:0/2598699532 shutdown_connections 2024-09-17T03:15:35.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.333+0000 7ff79d3ed640 1 -- 172.21.15.87:0/2598699532 wait complete. 2024-09-17T03:15:35.346 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-17T03:15:35.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.465+0000 7f8b3c6d3640 1 -- 172.21.15.87:0/3393274014 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b34074fb0 msgr2=0x7f8b340c4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:35.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.465+0000 7f8b3c6d3640 1 --2- 172.21.15.87:0/3393274014 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b34074fb0 0x7f8b340c4f00 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c005fd0 tx=0x7f8b2c02aa90 comp rx=0 tx=0).stop 2024-09-17T03:15:35.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.466+0000 7f8b3c6d3640 1 -- 172.21.15.87:0/3393274014 shutdown_connections 2024-09-17T03:15:35.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.466+0000 7f8b3c6d3640 1 --2- 172.21.15.87:0/3393274014 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b34074fb0 0x7f8b340c4f00 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:35.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.466+0000 7f8b3c6d3640 1 -- 172.21.15.87:0/3393274014 >> 172.21.15.87:0/3393274014 conn(0x7f8b3406ee70 msgr2=0x7f8b3406f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:35.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.466+0000 7f8b3c6d3640 1 -- 172.21.15.87:0/3393274014 shutdown_connections 2024-09-17T03:15:35.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.466+0000 7f8b3c6d3640 1 -- 172.21.15.87:0/3393274014 wait complete. 2024-09-17T03:15:35.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.467+0000 7f8b3c6d3640 1 Processor -- start 2024-09-17T03:15:35.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.467+0000 7f8b3c6d3640 1 -- start start 2024-09-17T03:15:35.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.469+0000 7f8b3c6d3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b340cc590 0x7f8b341ef6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:35.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.469+0000 7f8b3c6d3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b3415f0c0 0x7f8b340c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:35.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.469+0000 7f8b3c6d3640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8b341f5660 0x7f8b341f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:35.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.469+0000 7f8b3c6d3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8b340c5440 con 0x7f8b340cc590 2024-09-17T03:15:35.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.469+0000 7f8b3c6d3640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3415f6e0 con 0x7f8b3415f0c0 2024-09-17T03:15:35.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.469+0000 7f8b3c6d3640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8b340c55c0 con 0x7f8b341f5660 2024-09-17T03:15:35.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.469+0000 7f8b3a46f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b340cc590 0x7f8b341ef6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:35.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.469+0000 7f8b3a46f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b340cc590 0x7f8b341ef6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35944/0 (socket says 172.21.15.87:35944) 2024-09-17T03:15:35.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.469+0000 7f8b3a46f640 1 -- 172.21.15.87:0/2289280240 learned_addr learned my addr 172.21.15.87:0/2289280240 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:35.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.469+0000 7f8b39c6e640 1 --2- 172.21.15.87:0/2289280240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b3415f0c0 0x7f8b340c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:35.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.469+0000 7f8b3ac70640 1 --2- 172.21.15.87:0/2289280240 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8b341f5660 0x7f8b341f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:35.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.469+0000 7f8b3a46f640 1 -- 172.21.15.87:0/2289280240 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8b341f5660 msgr2=0x7f8b341f5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:35.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.469+0000 7f8b3a46f640 1 --2- 172.21.15.87:0/2289280240 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8b341f5660 0x7f8b341f5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:35.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.469+0000 7f8b3a46f640 1 -- 172.21.15.87:0/2289280240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b3415f0c0 msgr2=0x7f8b340c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:35.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.469+0000 7f8b3a46f640 1 --2- 172.21.15.87:0/2289280240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b3415f0c0 0x7f8b340c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:35.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.469+0000 7f8b3a46f640 1 -- 172.21.15.87:0/2289280240 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b34159790 con 0x7f8b340cc590 2024-09-17T03:15:35.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.469+0000 7f8b3a46f640 1 --2- 172.21.15.87:0/2289280240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b340cc590 0x7f8b341ef6e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000d9c0 tx=0x7f8b3000de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:35.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.469+0000 7f8b2affd640 1 -- 172.21.15.87:0/2289280240 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b3000ed40 con 0x7f8b340cc590 2024-09-17T03:15:35.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.469+0000 7f8b2affd640 1 -- 172.21.15.87:0/2289280240 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b30004020 con 0x7f8b340cc590 2024-09-17T03:15:35.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.469+0000 7f8b3c6d3640 1 -- 172.21.15.87:0/2289280240 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b3415afd0 con 0x7f8b340cc590 2024-09-17T03:15:35.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.469+0000 7f8b2affd640 1 -- 172.21.15.87:0/2289280240 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b300035f0 con 0x7f8b340cc590 2024-09-17T03:15:35.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.469+0000 7f8b3c6d3640 1 -- 172.21.15.87:0/2289280240 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b340c9c10 con 0x7f8b340cc590 2024-09-17T03:15:35.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.470+0000 7f8b3c6d3640 1 -- 172.21.15.87:0/2289280240 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b340b3960 con 0x7f8b340cc590 2024-09-17T03:15:35.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.471+0000 7f8b2affd640 1 -- 172.21.15.87:0/2289280240 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8b30003790 con 0x7f8b340cc590 2024-09-17T03:15:35.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.471+0000 7f8b2affd640 1 --2- 172.21.15.87:0/2289280240 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8b18078b90 0x7f8b1807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:35.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.472+0000 7f8b2affd640 1 -- 172.21.15.87:0/2289280240 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 4857+0+0 (secure 0 0 0) 0x7f8b3009c870 con 0x7f8b340cc590 2024-09-17T03:15:35.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.472+0000 7f8b39c6e640 1 --2- 172.21.15.87:0/2289280240 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8b18078b90 0x7f8b1807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:35.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.472+0000 7f8b39c6e640 1 --2- 172.21.15.87:0/2289280240 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8b18078b90 0x7f8b1807b0a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8b3415b700 tx=0x7f8b24009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:35.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.476+0000 7f8b2affd640 1 -- 172.21.15.87:0/2289280240 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f8b30064a40 con 0x7f8b340cc590 2024-09-17T03:15:35.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.719+0000 7f8b3c6d3640 1 -- 172.21.15.87:0/2289280240 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f8b340c4f00 con 0x7f8b340cc590 2024-09-17T03:15:35.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.719+0000 7f8b2affd640 1 -- 172.21.15.87:0/2289280240 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5298 (secure 0 0 0) 0x7f8b30069470 con 0x7f8b340cc590 2024-09-17T03:15:35.719 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:35.719 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-17T03:15:35.719 INFO:teuthology.orchestra.run.smithi087.stdout: "auth_dump": [ 2024-09-17T03:15:35.719 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:15:35.719 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.a", 2024-09-17T03:15:35.719 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDH8+hmLoKqFBAAl7DDFJqmrUwbDOL3J8wysA==", 2024-09-17T03:15:35.719 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:15:35.719 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-09-17T03:15:35.719 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-09-17T03:15:35.720 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-09-17T03:15:35.720 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:15:35.720 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:35.720 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:35.720 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:15:35.720 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.b", 2024-09-17T03:15:35.720 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDH8+hmOziYIRAA2hVbfPg5KtY+dF/o1HzAFA==", 2024-09-17T03:15:35.720 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:15:35.720 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-09-17T03:15:35.720 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-09-17T03:15:35.720 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-09-17T03:15:35.720 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:15:35.720 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:35.721 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:35.721 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:15:35.721 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.c", 2024-09-17T03:15:35.721 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDH8+hm+QtNHBAANIlkESYiJ80tcoyvu5KZLA==", 2024-09-17T03:15:35.721 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:15:35.721 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-09-17T03:15:35.721 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-09-17T03:15:35.721 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-09-17T03:15:35.721 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:15:35.721 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:35.721 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:35.721 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:15:35.721 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mds.d", 2024-09-17T03:15:35.722 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDH8+hmRUxHKRAARWfxi+PE89hD4fMhCdbnxQ==", 2024-09-17T03:15:35.722 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:15:35.722 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-09-17T03:15:35.722 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-09-17T03:15:35.722 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-09-17T03:15:35.722 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:15:35.722 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:35.722 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:35.722 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:15:35.722 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.0", 2024-09-17T03:15:35.722 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDL8+hm7Rx0ChAA8g5yLI7cCaX55jkLjNkzQA==", 2024-09-17T03:15:35.722 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:15:35.722 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-09-17T03:15:35.722 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-09-17T03:15:35.723 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:15:35.723 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:35.723 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:35.723 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:15:35.723 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.1", 2024-09-17T03:15:35.723 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDO8+hmQzq8KxAA5LsvmZb7P2FIPXd9wc23Ig==", 2024-09-17T03:15:35.723 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:15:35.723 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-09-17T03:15:35.723 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-09-17T03:15:35.723 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:15:35.723 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:35.723 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:35.723 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:15:35.724 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.2", 2024-09-17T03:15:35.724 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDS8+hm0jabEBAAYJIY3OgCoJ4EcYBJr89xSQ==", 2024-09-17T03:15:35.724 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:15:35.724 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-09-17T03:15:35.724 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-09-17T03:15:35.724 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:15:35.724 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:35.724 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:35.724 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:15:35.724 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.3", 2024-09-17T03:15:35.724 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDV8+hmSrsMIhAAWW1HvxXttwHNUJT5NxsPYA==", 2024-09-17T03:15:35.724 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:15:35.724 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-09-17T03:15:35.724 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-09-17T03:15:35.725 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:15:35.725 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:35.725 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:35.725 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:15:35.725 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.4", 2024-09-17T03:15:35.725 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDb8+hmg54IOxAAKQ8plXDnZxp7QlGZcNOVXg==", 2024-09-17T03:15:35.725 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:15:35.725 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-09-17T03:15:35.725 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-09-17T03:15:35.725 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:15:35.725 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:35.725 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:35.725 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:15:35.726 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.5", 2024-09-17T03:15:35.726 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDf8+hmUT1GLhAA6NQyWAThxgYjnf2X+PFE0A==", 2024-09-17T03:15:35.726 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:15:35.726 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-09-17T03:15:35.726 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-09-17T03:15:35.726 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:15:35.726 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:35.726 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:35.726 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:15:35.726 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.6", 2024-09-17T03:15:35.726 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDj8+hmjDo2IRAA6UyXPCnTHBilelz+S6IWmQ==", 2024-09-17T03:15:35.726 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:15:35.727 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-09-17T03:15:35.727 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-09-17T03:15:35.727 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:15:35.727 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:35.727 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:35.727 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:15:35.727 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "osd.7", 2024-09-17T03:15:35.727 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDm8+hmJKtHMhAA56cm6L9B9GLL48bRfjBnLg==", 2024-09-17T03:15:35.727 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:15:35.727 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow profile osd", 2024-09-17T03:15:35.727 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile osd", 2024-09-17T03:15:35.727 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:15:35.727 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:35.728 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:35.728 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:15:35.728 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.0", 2024-09-17T03:15:35.728 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDH8+hmLskNLhAAd2M5ynGP8W6Pim11F18wYA==", 2024-09-17T03:15:35.728 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:15:35.728 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-09-17T03:15:35.728 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow r", 2024-09-17T03:15:35.728 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rw pool=cephfs_data" 2024-09-17T03:15:35.728 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:35.728 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:35.728 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:15:35.728 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.1", 2024-09-17T03:15:35.728 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDH8+hm+aGqNRAAUG+G7uzrYQ90DHV08QvWKQ==", 2024-09-17T03:15:35.729 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:15:35.729 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-09-17T03:15:35.729 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-09-17T03:15:35.729 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-09-17T03:15:35.729 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-09-17T03:15:35.729 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:35.729 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:35.729 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:15:35.729 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.2", 2024-09-17T03:15:35.729 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDH8+hmqYfNMRAAzS0etgXaryNA6diH8MM4vQ==", 2024-09-17T03:15:35.729 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:15:35.729 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-09-17T03:15:35.729 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-09-17T03:15:35.730 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-09-17T03:15:35.730 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-09-17T03:15:35.730 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:35.730 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:35.730 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:15:35.730 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.3", 2024-09-17T03:15:35.730 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDH8+hmKQl7ORAAV04v5s/X/fn24bp8kLIRsA==", 2024-09-17T03:15:35.730 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:15:35.730 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow", 2024-09-17T03:15:35.730 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow r", 2024-09-17T03:15:35.730 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow rw", 2024-09-17T03:15:35.730 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow rwx" 2024-09-17T03:15:35.730 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:35.730 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:35.731 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:15:35.731 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "client.admin", 2024-09-17T03:15:35.731 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDG8+hmlf1wMhAA0b5O35T4hJS7YPD/EBn0WQ==", 2024-09-17T03:15:35.731 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:15:35.731 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-09-17T03:15:35.731 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr": "allow *", 2024-09-17T03:15:35.731 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow *", 2024-09-17T03:15:35.731 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:15:35.731 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:35.731 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:35.731 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:15:35.731 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.x", 2024-09-17T03:15:35.731 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDH8+hmGA/bEBAA7l5te6R2YeXND6keGQpVSw==", 2024-09-17T03:15:35.732 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:15:35.732 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-09-17T03:15:35.732 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-09-17T03:15:35.732 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:15:35.732 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:35.732 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:35.732 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:15:35.732 INFO:teuthology.orchestra.run.smithi087.stdout: "entity": "mgr.y", 2024-09-17T03:15:35.732 INFO:teuthology.orchestra.run.smithi087.stdout: "key": "AQDH8+hmsBkdDBAA4bwvaWSWANblx5JF75wTQQ==", 2024-09-17T03:15:35.732 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": { 2024-09-17T03:15:35.732 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": "allow *", 2024-09-17T03:15:35.733 INFO:teuthology.orchestra.run.smithi087.stdout: "mon": "allow profile mgr", 2024-09-17T03:15:35.733 INFO:teuthology.orchestra.run.smithi087.stdout: "osd": "allow *" 2024-09-17T03:15:35.733 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:35.733 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:35.733 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-09-17T03:15:35.733 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-17T03:15:35.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.721+0000 7f8b3c6d3640 1 -- 172.21.15.87:0/2289280240 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8b18078b90 msgr2=0x7f8b1807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:35.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.721+0000 7f8b3c6d3640 1 --2- 172.21.15.87:0/2289280240 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8b18078b90 0x7f8b1807b0a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8b3415b700 tx=0x7f8b24009040 comp rx=0 tx=0).stop 2024-09-17T03:15:35.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.721+0000 7f8b3c6d3640 1 -- 172.21.15.87:0/2289280240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b340cc590 msgr2=0x7f8b341ef6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:35.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.721+0000 7f8b3c6d3640 1 --2- 172.21.15.87:0/2289280240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b340cc590 0x7f8b341ef6e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8b3000d9c0 tx=0x7f8b3000de80 comp rx=0 tx=0).stop 2024-09-17T03:15:35.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.721+0000 7f8b3c6d3640 1 -- 172.21.15.87:0/2289280240 shutdown_connections 2024-09-17T03:15:35.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.721+0000 7f8b3c6d3640 1 --2- 172.21.15.87:0/2289280240 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8b18078b90 0x7f8b1807b0a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:35.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.721+0000 7f8b3c6d3640 1 --2- 172.21.15.87:0/2289280240 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f8b341f5660 0x7f8b341f5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:35.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.721+0000 7f8b3c6d3640 1 --2- 172.21.15.87:0/2289280240 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f8b3415f0c0 0x7f8b340c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:35.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.721+0000 7f8b3c6d3640 1 --2- 172.21.15.87:0/2289280240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b340cc590 0x7f8b341ef6e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:35.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.721+0000 7f8b3c6d3640 1 -- 172.21.15.87:0/2289280240 >> 172.21.15.87:0/2289280240 conn(0x7f8b3406ee70 msgr2=0x7f8b3406f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:35.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.721+0000 7f8b3c6d3640 1 -- 172.21.15.87:0/2289280240 shutdown_connections 2024-09-17T03:15:35.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.721+0000 7f8b3c6d3640 1 -- 172.21.15.87:0/2289280240 wait complete. 2024-09-17T03:15:35.736 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-17T03:15:35.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.849+0000 7f13cf5cc640 1 -- 172.21.15.87:0/3654794671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13c80b7f10 msgr2=0x7f13c815c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:35.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.849+0000 7f13cf5cc640 1 --2- 172.21.15.87:0/3654794671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13c80b7f10 0x7f13c815c5f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f13c4009830 tx=0x7f13c402a480 comp rx=0 tx=0).stop 2024-09-17T03:15:35.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.849+0000 7f13cf5cc640 1 -- 172.21.15.87:0/3654794671 shutdown_connections 2024-09-17T03:15:35.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.849+0000 7f13cf5cc640 1 --2- 172.21.15.87:0/3654794671 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13c80b3d60 0x7f13c80b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:35.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.849+0000 7f13cf5cc640 1 --2- 172.21.15.87:0/3654794671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13c80b7f10 0x7f13c815c5f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:35.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.849+0000 7f13cf5cc640 1 -- 172.21.15.87:0/3654794671 >> 172.21.15.87:0/3654794671 conn(0x7f13c8069330 msgr2=0x7f13c80ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:35.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.849+0000 7f13cf5cc640 1 -- 172.21.15.87:0/3654794671 shutdown_connections 2024-09-17T03:15:35.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.849+0000 7f13cf5cc640 1 -- 172.21.15.87:0/3654794671 wait complete. 2024-09-17T03:15:35.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.850+0000 7f13cf5cc640 1 Processor -- start 2024-09-17T03:15:35.850 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.851+0000 7f13cf5cc640 1 -- start start 2024-09-17T03:15:35.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.852+0000 7f13cf5cc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13c80710e0 0x7f13c814b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:35.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.852+0000 7f13cf5cc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13c80b3d60 0x7f13c814c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:35.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.852+0000 7f13cf5cc640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13c80b7f10 0x7f13c814cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:35.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.852+0000 7f13cf5cc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f13c80b7b10 con 0x7f13c80710e0 2024-09-17T03:15:35.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.852+0000 7f13cf5cc640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f13c80d1c30 con 0x7f13c80b3d60 2024-09-17T03:15:35.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.852+0000 7f13cf5cc640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f13c80b7c90 con 0x7f13c80b7f10 2024-09-17T03:15:35.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.852+0000 7f13cd368640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13c80710e0 0x7f13c814b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:35.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.852+0000 7f13cd368640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13c80710e0 0x7f13c814b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35966/0 (socket says 172.21.15.87:35966) 2024-09-17T03:15:35.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.852+0000 7f13cd368640 1 -- 172.21.15.87:0/867612459 learned_addr learned my addr 172.21.15.87:0/867612459 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:35.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.852+0000 7f13cd368640 1 -- 172.21.15.87:0/867612459 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13c80b7f10 msgr2=0x7f13c814cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:35.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.852+0000 7f13cd368640 1 --2- 172.21.15.87:0/867612459 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13c80b7f10 0x7f13c814cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:35.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.852+0000 7f13cd368640 1 -- 172.21.15.87:0/867612459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13c80b3d60 msgr2=0x7f13c814c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:35.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.852+0000 7f13cd368640 1 --2- 172.21.15.87:0/867612459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13c80b3d60 0x7f13c814c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:35.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.852+0000 7f13cd368640 1 -- 172.21.15.87:0/867612459 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c814bee0 con 0x7f13c80710e0 2024-09-17T03:15:35.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.852+0000 7f13cd368640 1 --2- 172.21.15.87:0/867612459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13c80710e0 0x7f13c814b9a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f13c4009950 tx=0x7f13c4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:35.854 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.853+0000 7f13bdffb640 1 -- 172.21.15.87:0/867612459 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4005c90 con 0x7f13c80710e0 2024-09-17T03:15:35.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.853+0000 7f13bdffb640 1 -- 172.21.15.87:0/867612459 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13c4031e50 con 0x7f13c80710e0 2024-09-17T03:15:35.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.853+0000 7f13cf5cc640 1 -- 172.21.15.87:0/867612459 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c80ca590 con 0x7f13c80710e0 2024-09-17T03:15:35.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.853+0000 7f13bdffb640 1 -- 172.21.15.87:0/867612459 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4005260 con 0x7f13c80710e0 2024-09-17T03:15:35.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.853+0000 7f13cf5cc640 1 -- 172.21.15.87:0/867612459 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c80cab50 con 0x7f13c80710e0 2024-09-17T03:15:35.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.854+0000 7f13cf5cc640 1 -- 172.21.15.87:0/867612459 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13c814dca0 con 0x7f13c80710e0 2024-09-17T03:15:35.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.854+0000 7f13bdffb640 1 -- 172.21.15.87:0/867612459 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f13c4005400 con 0x7f13c80710e0 2024-09-17T03:15:35.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.855+0000 7f13bdffb640 1 --2- 172.21.15.87:0/867612459 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f13ac078b90 0x7f13ac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:35.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.855+0000 7f13bdffb640 1 -- 172.21.15.87:0/867612459 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 4857+0+0 (secure 0 0 0) 0x7f13c40bbb10 con 0x7f13c80710e0 2024-09-17T03:15:35.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.855+0000 7f13ccb67640 1 --2- 172.21.15.87:0/867612459 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f13ac078b90 0x7f13ac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:35.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.856+0000 7f13ccb67640 1 --2- 172.21.15.87:0/867612459 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f13ac078b90 0x7f13ac07b0a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f13b80058c0 tx=0x7f13b8005850 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:35.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:35.861+0000 7f13bdffb640 1 -- 172.21.15.87:0/867612459 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f13c403f030 con 0x7f13c80710e0 2024-09-17T03:15:36.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.035+0000 7f13cf5cc640 1 -- 172.21.15.87:0/867612459 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f13c8071e30 con 0x7f13c80710e0 2024-09-17T03:15:36.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.035+0000 7f13bdffb640 1 -- 172.21.15.87:0/867612459 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v46) ==== 84+0+65 (secure 0 0 0) 0x7f13c4084330 con 0x7f13c80710e0 2024-09-17T03:15:36.035 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:36.035 INFO:teuthology.orchestra.run.smithi087.stdout:[ 2024-09-17T03:15:36.035 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-09-17T03:15:36.035 INFO:teuthology.orchestra.run.smithi087.stdout: "poolnum": 1, 2024-09-17T03:15:36.035 INFO:teuthology.orchestra.run.smithi087.stdout: "poolname": ".mgr" 2024-09-17T03:15:36.035 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:36.035 INFO:teuthology.orchestra.run.smithi087.stdout:] 2024-09-17T03:15:36.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.037+0000 7f13cf5cc640 1 -- 172.21.15.87:0/867612459 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f13ac078b90 msgr2=0x7f13ac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:36.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.037+0000 7f13cf5cc640 1 --2- 172.21.15.87:0/867612459 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f13ac078b90 0x7f13ac07b0a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f13b80058c0 tx=0x7f13b8005850 comp rx=0 tx=0).stop 2024-09-17T03:15:36.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.037+0000 7f13cf5cc640 1 -- 172.21.15.87:0/867612459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13c80710e0 msgr2=0x7f13c814b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:36.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.037+0000 7f13cf5cc640 1 --2- 172.21.15.87:0/867612459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13c80710e0 0x7f13c814b9a0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f13c4009950 tx=0x7f13c4033c20 comp rx=0 tx=0).stop 2024-09-17T03:15:36.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.037+0000 7f13cf5cc640 1 -- 172.21.15.87:0/867612459 shutdown_connections 2024-09-17T03:15:36.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.037+0000 7f13cf5cc640 1 --2- 172.21.15.87:0/867612459 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f13ac078b90 0x7f13ac07b0a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:36.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.037+0000 7f13cf5cc640 1 --2- 172.21.15.87:0/867612459 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f13c80b7f10 0x7f13c814cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:36.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.037+0000 7f13cf5cc640 1 --2- 172.21.15.87:0/867612459 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f13c80b3d60 0x7f13c814c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:36.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.037+0000 7f13cf5cc640 1 --2- 172.21.15.87:0/867612459 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13c80710e0 0x7f13c814b9a0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:36.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.037+0000 7f13cf5cc640 1 -- 172.21.15.87:0/867612459 >> 172.21.15.87:0/867612459 conn(0x7f13c8069330 msgr2=0x7f13c80d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:36.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.037+0000 7f13cf5cc640 1 -- 172.21.15.87:0/867612459 shutdown_connections 2024-09-17T03:15:36.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.037+0000 7f13cf5cc640 1 -- 172.21.15.87:0/867612459 wait complete. 2024-09-17T03:15:36.048 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-17T03:15:36.048 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-17T03:15:36.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.166+0000 7f34bd3da640 1 -- 172.21.15.87:0/1447552402 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f34b80b3ae0 msgr2=0x7f34b80b95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:36.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.166+0000 7f34bd3da640 1 --2- 172.21.15.87:0/1447552402 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f34b80b3ae0 0x7f34b80b95e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f34a8005fd0 tx=0x7f34a802a5d0 comp rx=0 tx=0).stop 2024-09-17T03:15:36.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.166+0000 7f34bd3da640 1 -- 172.21.15.87:0/1447552402 shutdown_connections 2024-09-17T03:15:36.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.166+0000 7f34bd3da640 1 --2- 172.21.15.87:0/1447552402 >> v2:172.21.15.149:3300/0 conn(0x7f34b8161dd0 0x7f34b80cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:36.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.166+0000 7f34bd3da640 1 --2- 172.21.15.87:0/1447552402 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f34b80b3ae0 0x7f34b80b95e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:36.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.166+0000 7f34bd3da640 1 -- 172.21.15.87:0/1447552402 >> 172.21.15.87:0/1447552402 conn(0x7f34b809d7e0 msgr2=0x7f34b808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:36.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.166+0000 7f34bd3da640 1 -- 172.21.15.87:0/1447552402 shutdown_connections 2024-09-17T03:15:36.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.166+0000 7f34bd3da640 1 -- 172.21.15.87:0/1447552402 wait complete. 2024-09-17T03:15:36.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.168+0000 7f34bd3da640 1 Processor -- start 2024-09-17T03:15:36.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.168+0000 7f34bd3da640 1 -- start start 2024-09-17T03:15:36.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.169+0000 7f34bd3da640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f34b80b3ae0 0x7f34b8159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:36.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.169+0000 7f34bd3da640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34b80d0d00 0x7f34b80cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:36.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.169+0000 7f34bd3da640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34b8161dd0 0x7f34b81f9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:36.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.169+0000 7f34bd3da640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f34b80d2750 con 0x7f34b80d0d00 2024-09-17T03:15:36.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.169+0000 7f34bd3da640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f34b80d25d0 con 0x7f34b8161dd0 2024-09-17T03:15:36.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.169+0000 7f34bd3da640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f34b80d28d0 con 0x7f34b80b3ae0 2024-09-17T03:15:36.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.169+0000 7f34b67fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34b80d0d00 0x7f34b80cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:36.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.169+0000 7f34b67fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34b80d0d00 0x7f34b80cab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35970/0 (socket says 172.21.15.87:35970) 2024-09-17T03:15:36.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.169+0000 7f34b67fc640 1 -- 172.21.15.87:0/1184017352 learned_addr learned my addr 172.21.15.87:0/1184017352 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:36.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.169+0000 7f34b6ffd640 1 --2- 172.21.15.87:0/1184017352 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f34b80b3ae0 0x7f34b8159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:36.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.169+0000 7f34b77fe640 1 --2- 172.21.15.87:0/1184017352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34b8161dd0 0x7f34b81f9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:36.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.170+0000 7f34b67fc640 1 -- 172.21.15.87:0/1184017352 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f34b80b3ae0 msgr2=0x7f34b8159f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:36.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.170+0000 7f34b67fc640 1 --2- 172.21.15.87:0/1184017352 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f34b80b3ae0 0x7f34b8159f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:36.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.170+0000 7f34b67fc640 1 -- 172.21.15.87:0/1184017352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34b8161dd0 msgr2=0x7f34b81f9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:36.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.170+0000 7f34b67fc640 1 --2- 172.21.15.87:0/1184017352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34b8161dd0 0x7f34b81f9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:36.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.170+0000 7f34b67fc640 1 -- 172.21.15.87:0/1184017352 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b814af20 con 0x7f34b80d0d00 2024-09-17T03:15:36.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.170+0000 7f34b67fc640 1 --2- 172.21.15.87:0/1184017352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34b80d0d00 0x7f34b80cab20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00ea20 tx=0x7f34ac00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:36.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.170+0000 7f349f7fe640 1 -- 172.21.15.87:0/1184017352 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac00ce40 con 0x7f34b80d0d00 2024-09-17T03:15:36.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.170+0000 7f349f7fe640 1 -- 172.21.15.87:0/1184017352 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34ac004020 con 0x7f34b80d0d00 2024-09-17T03:15:36.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.170+0000 7f34bd3da640 1 -- 172.21.15.87:0/1184017352 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b814a160 con 0x7f34b80d0d00 2024-09-17T03:15:36.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.170+0000 7f349f7fe640 1 -- 172.21.15.87:0/1184017352 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ac004470 con 0x7f34b80d0d00 2024-09-17T03:15:36.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.170+0000 7f34bd3da640 1 -- 172.21.15.87:0/1184017352 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b814a720 con 0x7f34b80d0d00 2024-09-17T03:15:36.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.171+0000 7f34bd3da640 1 -- 172.21.15.87:0/1184017352 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b80d2dc0 con 0x7f34b80d0d00 2024-09-17T03:15:36.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.172+0000 7f349f7fe640 1 -- 172.21.15.87:0/1184017352 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f34ac0035f0 con 0x7f34b80d0d00 2024-09-17T03:15:36.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.172+0000 7f349f7fe640 1 --2- 172.21.15.87:0/1184017352 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f34a4078b90 0x7f34a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:36.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.172+0000 7f349f7fe640 1 -- 172.21.15.87:0/1184017352 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 4857+0+0 (secure 0 0 0) 0x7f34ac09b440 con 0x7f34b80d0d00 2024-09-17T03:15:36.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.172+0000 7f34b6ffd640 1 --2- 172.21.15.87:0/1184017352 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f34a4078b90 0x7f34a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:36.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.173+0000 7f34b6ffd640 1 --2- 172.21.15.87:0/1184017352 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f34a4078b90 0x7f34a407b0a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f34a8033000 tx=0x7f34a8004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:36.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.177+0000 7f349f7fe640 1 -- 172.21.15.87:0/1184017352 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f34ac063500 con 0x7f34b80d0d00 2024-09-17T03:15:36.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:36.351+0000 7f34bd3da640 1 -- 172.21.15.87:0/1184017352 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f34b815fe60 con 0x7f34b80d0d00 2024-09-17T03:15:37.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.334+0000 7f349f7fe640 1 -- 172.21.15.87:0/1184017352 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v47) ==== 138+0+0 (secure 0 0 0) 0x7f34ac067f30 con 0x7f34b80d0d00 2024-09-17T03:15:37.333 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' created 2024-09-17T03:15:37.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.337+0000 7f34bd3da640 1 -- 172.21.15.87:0/1184017352 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f34a4078b90 msgr2=0x7f34a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:37.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.337+0000 7f34bd3da640 1 --2- 172.21.15.87:0/1184017352 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f34a4078b90 0x7f34a407b0a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f34a8033000 tx=0x7f34a8004730 comp rx=0 tx=0).stop 2024-09-17T03:15:37.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.337+0000 7f34bd3da640 1 -- 172.21.15.87:0/1184017352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34b80d0d00 msgr2=0x7f34b80cab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:37.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.337+0000 7f34bd3da640 1 --2- 172.21.15.87:0/1184017352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34b80d0d00 0x7f34b80cab20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f34ac00ea20 tx=0x7f34ac00eee0 comp rx=0 tx=0).stop 2024-09-17T03:15:37.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.338+0000 7f34bd3da640 1 -- 172.21.15.87:0/1184017352 shutdown_connections 2024-09-17T03:15:37.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.338+0000 7f34bd3da640 1 --2- 172.21.15.87:0/1184017352 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f34a4078b90 0x7f34a407b0a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:37.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.338+0000 7f34bd3da640 1 --2- 172.21.15.87:0/1184017352 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f34b8161dd0 0x7f34b81f9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:37.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.338+0000 7f34bd3da640 1 --2- 172.21.15.87:0/1184017352 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34b80d0d00 0x7f34b80cab20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:37.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.338+0000 7f34bd3da640 1 --2- 172.21.15.87:0/1184017352 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f34b80b3ae0 0x7f34b8159f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:37.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.338+0000 7f34bd3da640 1 -- 172.21.15.87:0/1184017352 >> 172.21.15.87:0/1184017352 conn(0x7f34b809d7e0 msgr2=0x7f34b815ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:37.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.338+0000 7f34bd3da640 1 -- 172.21.15.87:0/1184017352 shutdown_connections 2024-09-17T03:15:37.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.338+0000 7f34bd3da640 1 -- 172.21.15.87:0/1184017352 wait complete. 2024-09-17T03:15:37.354 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-17T03:15:37.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.481+0000 7f976b912640 1 -- 172.21.15.87:0/1995154888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f976415f650 msgr2=0x7f97641660c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:37.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.481+0000 7f976b912640 1 --2- 172.21.15.87:0/1995154888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f976415f650 0x7f97641660c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f975c005fd0 tx=0x7f975c02a8b0 comp rx=0 tx=0).stop 2024-09-17T03:15:37.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.481+0000 7f976b912640 1 -- 172.21.15.87:0/1995154888 shutdown_connections 2024-09-17T03:15:37.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.481+0000 7f976b912640 1 --2- 172.21.15.87:0/1995154888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f976415f650 0x7f97641660c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:37.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.481+0000 7f976b912640 1 -- 172.21.15.87:0/1995154888 >> 172.21.15.87:0/1995154888 conn(0x7f97640b1c40 msgr2=0x7f97640b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:37.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.481+0000 7f976b912640 1 -- 172.21.15.87:0/1995154888 shutdown_connections 2024-09-17T03:15:37.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.481+0000 7f976b912640 1 -- 172.21.15.87:0/1995154888 wait complete. 2024-09-17T03:15:37.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.483+0000 7f976b912640 1 Processor -- start 2024-09-17T03:15:37.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.483+0000 7f976b912640 1 -- start start 2024-09-17T03:15:37.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.484+0000 7f976b912640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9764152490 0x7f97641445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:37.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.484+0000 7f976b912640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f976415fb50 0x7f976414d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:37.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.484+0000 7f976b912640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97641f5ab0 0x7f97641f5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:37.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.484+0000 7f976b912640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9764166990 con 0x7f9764152490 2024-09-17T03:15:37.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.484+0000 7f976b912640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f9764166810 con 0x7f976415fb50 2024-09-17T03:15:37.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.484+0000 7f976b912640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f9764166b10 con 0x7f97641f5ab0 2024-09-17T03:15:37.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.484+0000 7f97696ae640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9764152490 0x7f97641445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:37.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.484+0000 7f97696ae640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9764152490 0x7f97641445b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35990/0 (socket says 172.21.15.87:35990) 2024-09-17T03:15:37.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.484+0000 7f97696ae640 1 -- 172.21.15.87:0/1927661489 learned_addr learned my addr 172.21.15.87:0/1927661489 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:37.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.484+0000 7f9768ead640 1 --2- 172.21.15.87:0/1927661489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f976415fb50 0x7f976414d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:37.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.485+0000 7f9769eaf640 1 --2- 172.21.15.87:0/1927661489 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97641f5ab0 0x7f97641f5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:37.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.485+0000 7f97696ae640 1 -- 172.21.15.87:0/1927661489 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97641f5ab0 msgr2=0x7f97641f5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:37.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.485+0000 7f97696ae640 1 --2- 172.21.15.87:0/1927661489 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97641f5ab0 0x7f97641f5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:37.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.485+0000 7f97696ae640 1 -- 172.21.15.87:0/1927661489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f976415fb50 msgr2=0x7f976414d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:37.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.485+0000 7f97696ae640 1 --2- 172.21.15.87:0/1927661489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f976415fb50 0x7f976414d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:37.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.485+0000 7f97696ae640 1 -- 172.21.15.87:0/1927661489 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97641f6400 con 0x7f9764152490 2024-09-17T03:15:37.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.485+0000 7f9768ead640 1 --2- 172.21.15.87:0/1927661489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f976415fb50 0x7f976414d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:15:37.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.485+0000 7f97696ae640 1 --2- 172.21.15.87:0/1927661489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9764152490 0x7f97641445b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f975400c3d0 tx=0x7f975400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:37.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.485+0000 7f9761ffb640 1 -- 172.21.15.87:0/1927661489 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754011790 con 0x7f9764152490 2024-09-17T03:15:37.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.485+0000 7f9761ffb640 1 -- 172.21.15.87:0/1927661489 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9754017070 con 0x7f9764152490 2024-09-17T03:15:37.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.485+0000 7f976b912640 1 -- 172.21.15.87:0/1927661489 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9764147500 con 0x7f9764152490 2024-09-17T03:15:37.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.485+0000 7f9761ffb640 1 -- 172.21.15.87:0/1927661489 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754011cc0 con 0x7f9764152490 2024-09-17T03:15:37.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.485+0000 7f976b912640 1 -- 172.21.15.87:0/1927661489 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9764147ac0 con 0x7f9764152490 2024-09-17T03:15:37.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.486+0000 7f976b912640 1 -- 172.21.15.87:0/1927661489 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9764072da0 con 0x7f9764152490 2024-09-17T03:15:37.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.487+0000 7f9761ffb640 1 -- 172.21.15.87:0/1927661489 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f975401c460 con 0x7f9764152490 2024-09-17T03:15:37.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.488+0000 7f9761ffb640 1 --2- 172.21.15.87:0/1927661489 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9750078b90 0x7f975007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:37.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.488+0000 7f9761ffb640 1 -- 172.21.15.87:0/1927661489 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5227+0+0 (secure 0 0 0) 0x7f975409a7f0 con 0x7f9764152490 2024-09-17T03:15:37.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.488+0000 7f9768ead640 1 --2- 172.21.15.87:0/1927661489 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9750078b90 0x7f975007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:37.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.489+0000 7f9768ead640 1 --2- 172.21.15.87:0/1927661489 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9750078b90 0x7f975007b0a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f976415d9e0 tx=0x7f9758008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:37.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.493+0000 7f9761ffb640 1 -- 172.21.15.87:0/1927661489 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f97641f6400 con 0x7f9764152490 2024-09-17T03:15:37.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:37.669+0000 7f976b912640 1 -- 172.21.15.87:0/1927661489 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9764155d40 con 0x7f9764152490 2024-09-17T03:15:38.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.342+0000 7f9761ffb640 1 -- 172.21.15.87:0/1927661489 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v48) ==== 170+0+0 (secure 0 0 0) 0x7f97640adfd0 con 0x7f9764152490 2024-09-17T03:15:38.341 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' created 2024-09-17T03:15:38.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.345+0000 7f976b912640 1 -- 172.21.15.87:0/1927661489 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9750078b90 msgr2=0x7f975007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:38.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.345+0000 7f976b912640 1 --2- 172.21.15.87:0/1927661489 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9750078b90 0x7f975007b0a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f976415d9e0 tx=0x7f9758008040 comp rx=0 tx=0).stop 2024-09-17T03:15:38.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.345+0000 7f976b912640 1 -- 172.21.15.87:0/1927661489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9764152490 msgr2=0x7f97641445b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:38.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.345+0000 7f976b912640 1 --2- 172.21.15.87:0/1927661489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9764152490 0x7f97641445b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f975400c3d0 tx=0x7f975400c890 comp rx=0 tx=0).stop 2024-09-17T03:15:38.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.345+0000 7f976b912640 1 -- 172.21.15.87:0/1927661489 shutdown_connections 2024-09-17T03:15:38.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.345+0000 7f976b912640 1 --2- 172.21.15.87:0/1927661489 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f9750078b90 0x7f975007b0a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:38.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.345+0000 7f976b912640 1 --2- 172.21.15.87:0/1927661489 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f97641f5ab0 0x7f97641f5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:38.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.345+0000 7f976b912640 1 --2- 172.21.15.87:0/1927661489 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f976415fb50 0x7f976414d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:38.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.345+0000 7f976b912640 1 --2- 172.21.15.87:0/1927661489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9764152490 0x7f97641445b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:38.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.345+0000 7f976b912640 1 -- 172.21.15.87:0/1927661489 >> 172.21.15.87:0/1927661489 conn(0x7f97640b1c40 msgr2=0x7f976415e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:38.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.346+0000 7f976b912640 1 -- 172.21.15.87:0/1927661489 shutdown_connections 2024-09-17T03:15:38.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.346+0000 7f976b912640 1 -- 172.21.15.87:0/1927661489 wait complete. 2024-09-17T03:15:38.368 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-17T03:15:38.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.486+0000 7feda4870640 1 -- 172.21.15.87:0/3910089172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda00b3bc0 msgr2=0x7feda00b3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:38.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.486+0000 7feda4870640 1 --2- 172.21.15.87:0/3910089172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda00b3bc0 0x7feda00b3fd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7feda00d1980 tx=0x7fed9802a980 comp rx=0 tx=0).stop 2024-09-17T03:15:38.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.486+0000 7feda4870640 1 -- 172.21.15.87:0/3910089172 shutdown_connections 2024-09-17T03:15:38.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.486+0000 7feda4870640 1 --2- 172.21.15.87:0/3910089172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda00b3bc0 0x7feda00b3fd0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:38.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.486+0000 7feda4870640 1 --2- 172.21.15.87:0/3910089172 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feda0073580 0x7feda0073990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:38.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.486+0000 7feda4870640 1 --2- 172.21.15.87:0/3910089172 >> v2:172.21.15.149:3300/0 conn(0x7feda0070070 0x7feda00cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:38.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.486+0000 7feda4870640 1 -- 172.21.15.87:0/3910089172 >> 172.21.15.87:0/3910089172 conn(0x7feda00b2550 msgr2=0x7feda00b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:38.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.486+0000 7feda4870640 1 -- 172.21.15.87:0/3910089172 shutdown_connections 2024-09-17T03:15:38.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.486+0000 7feda4870640 1 -- 172.21.15.87:0/3910089172 wait complete. 2024-09-17T03:15:38.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.488+0000 7feda4870640 1 Processor -- start 2024-09-17T03:15:38.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.488+0000 7feda4870640 1 -- start start 2024-09-17T03:15:38.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.489+0000 7feda4870640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feda0070070 0x7feda013f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:38.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.489+0000 7feda4870640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feda0073580 0x7feda01597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:38.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.489+0000 7feda4870640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda00b3bc0 0x7feda0141050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:38.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.489+0000 7feda4870640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7feda00b4510 con 0x7feda00b3bc0 2024-09-17T03:15:38.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.489+0000 7feda4870640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7feda00cd190 con 0x7feda0073580 2024-09-17T03:15:38.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.489+0000 7feda4870640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7feda00b4690 con 0x7feda0070070 2024-09-17T03:15:38.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.489+0000 7fed9ed9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda00b3bc0 0x7feda0141050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:38.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.489+0000 7fed9ed9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda00b3bc0 0x7feda0141050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36002/0 (socket says 172.21.15.87:36002) 2024-09-17T03:15:38.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.489+0000 7fed9ed9d640 1 -- 172.21.15.87:0/662954298 learned_addr learned my addr 172.21.15.87:0/662954298 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:38.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.490+0000 7fed9e59c640 1 --2- 172.21.15.87:0/662954298 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feda0070070 0x7feda013f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:38.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.490+0000 7fed95d9b640 1 --2- 172.21.15.87:0/662954298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feda0073580 0x7feda01597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:38.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.490+0000 7fed9ed9d640 1 -- 172.21.15.87:0/662954298 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feda0070070 msgr2=0x7feda013f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:38.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.490+0000 7fed9ed9d640 1 --2- 172.21.15.87:0/662954298 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feda0070070 0x7feda013f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:38.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.490+0000 7fed9ed9d640 1 -- 172.21.15.87:0/662954298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feda0073580 msgr2=0x7feda01597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:38.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.490+0000 7fed9ed9d640 1 --2- 172.21.15.87:0/662954298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feda0073580 0x7feda01597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:38.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.490+0000 7fed9ed9d640 1 -- 172.21.15.87:0/662954298 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda0141c00 con 0x7feda00b3bc0 2024-09-17T03:15:38.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.490+0000 7fed9ed9d640 1 --2- 172.21.15.87:0/662954298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda00b3bc0 0x7feda0141050 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fed98033040 tx=0x7fed98003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:38.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.490+0000 7fed977fe640 1 -- 172.21.15.87:0/662954298 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98048070 con 0x7feda00b3bc0 2024-09-17T03:15:38.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.490+0000 7fed977fe640 1 -- 172.21.15.87:0/662954298 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed98004350 con 0x7feda00b3bc0 2024-09-17T03:15:38.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.490+0000 7feda4870640 1 -- 172.21.15.87:0/662954298 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda0141ee0 con 0x7feda00b3bc0 2024-09-17T03:15:38.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.490+0000 7fed977fe640 1 -- 172.21.15.87:0/662954298 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98034040 con 0x7feda00b3bc0 2024-09-17T03:15:38.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.490+0000 7feda4870640 1 -- 172.21.15.87:0/662954298 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda0158fa0 con 0x7feda00b3bc0 2024-09-17T03:15:38.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.491+0000 7feda4870640 1 -- 172.21.15.87:0/662954298 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feda0141970 con 0x7feda00b3bc0 2024-09-17T03:15:38.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.492+0000 7fed977fe640 1 -- 172.21.15.87:0/662954298 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fed9804b070 con 0x7feda00b3bc0 2024-09-17T03:15:38.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.492+0000 7fed977fe640 1 --2- 172.21.15.87:0/662954298 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fed80078b90 0x7fed8007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:38.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.492+0000 7fed977fe640 1 -- 172.21.15.87:0/662954298 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5609+0+0 (secure 0 0 0) 0x7fed980bdc40 con 0x7feda00b3bc0 2024-09-17T03:15:38.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.492+0000 7fed9e59c640 1 --2- 172.21.15.87:0/662954298 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fed80078b90 0x7fed8007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:38.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.493+0000 7fed9e59c640 1 --2- 172.21.15.87:0/662954298 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fed80078b90 0x7fed8007b0a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fed90005fd0 tx=0x7fed9000c460 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:38.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.497+0000 7fed977fe640 1 -- 172.21.15.87:0/662954298 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fed98086170 con 0x7feda00b3bc0 2024-09-17T03:15:38.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:38.706+0000 7feda4870640 1 -- 172.21.15.87:0/662954298 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7feda0073990 con 0x7feda00b3bc0 2024-09-17T03:15:39.055 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:15:39.058 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:15:39.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.344+0000 7fed977fe640 1 -- 172.21.15.87:0/662954298 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v18) ==== 172+0+0 (secure 0 0 0) 0x7fed98085a70 con 0x7feda00b3bc0 2024-09-17T03:15:39.343 INFO:teuthology.orchestra.run.smithi087.stderr:new fs with metadata pool 7 and data pool 8 2024-09-17T03:15:39.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.347+0000 7feda4870640 1 -- 172.21.15.87:0/662954298 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fed80078b90 msgr2=0x7fed8007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:39.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.347+0000 7feda4870640 1 --2- 172.21.15.87:0/662954298 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fed80078b90 0x7fed8007b0a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fed90005fd0 tx=0x7fed9000c460 comp rx=0 tx=0).stop 2024-09-17T03:15:39.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.347+0000 7feda4870640 1 -- 172.21.15.87:0/662954298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda00b3bc0 msgr2=0x7feda0141050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:39.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.347+0000 7feda4870640 1 --2- 172.21.15.87:0/662954298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda00b3bc0 0x7feda0141050 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fed98033040 tx=0x7fed98003ef0 comp rx=0 tx=0).stop 2024-09-17T03:15:39.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.347+0000 7feda4870640 1 -- 172.21.15.87:0/662954298 shutdown_connections 2024-09-17T03:15:39.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.347+0000 7feda4870640 1 --2- 172.21.15.87:0/662954298 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fed80078b90 0x7fed8007b0a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:39.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.347+0000 7feda4870640 1 --2- 172.21.15.87:0/662954298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feda00b3bc0 0x7feda0141050 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:39.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.347+0000 7feda4870640 1 --2- 172.21.15.87:0/662954298 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7feda0073580 0x7feda01597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:39.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.347+0000 7feda4870640 1 --2- 172.21.15.87:0/662954298 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7feda0070070 0x7feda013f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:39.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.347+0000 7feda4870640 1 -- 172.21.15.87:0/662954298 >> 172.21.15.87:0/662954298 conn(0x7feda00b2550 msgr2=0x7feda00b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:39.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.348+0000 7feda4870640 1 -- 172.21.15.87:0/662954298 shutdown_connections 2024-09-17T03:15:39.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.348+0000 7feda4870640 1 -- 172.21.15.87:0/662954298 wait complete. 2024-09-17T03:15:39.372 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:15:39.485 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.486+0000 7f30bb7e6640 1 -- 172.21.15.87:0/89236382 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30b415e8f0 msgr2=0x7f30b415ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:39.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.486+0000 7f30bb7e6640 1 --2- 172.21.15.87:0/89236382 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30b415e8f0 0x7f30b415ed00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f30a4005fd0 tx=0x7f30a402a540 comp rx=0 tx=0).stop 2024-09-17T03:15:39.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.486+0000 7f30bb7e6640 1 -- 172.21.15.87:0/89236382 shutdown_connections 2024-09-17T03:15:39.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.486+0000 7f30bb7e6640 1 --2- 172.21.15.87:0/89236382 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30b415e8f0 0x7f30b415ed00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:39.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.486+0000 7f30bb7e6640 1 --2- 172.21.15.87:0/89236382 >> v2:172.21.15.149:3300/0 conn(0x7f30b40af680 0x7f30b415acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:39.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.486+0000 7f30bb7e6640 1 -- 172.21.15.87:0/89236382 >> 172.21.15.87:0/89236382 conn(0x7f30b406ee70 msgr2=0x7f30b406f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:39.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.486+0000 7f30bb7e6640 1 -- 172.21.15.87:0/89236382 shutdown_connections 2024-09-17T03:15:39.486 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.487+0000 7f30bb7e6640 1 -- 172.21.15.87:0/89236382 wait complete. 2024-09-17T03:15:39.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.488+0000 7f30bb7e6640 1 Processor -- start 2024-09-17T03:15:39.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.488+0000 7f30bb7e6640 1 -- start start 2024-09-17T03:15:39.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.489+0000 7f30bb7e6640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30b40af680 0x7f30b40b5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:39.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.490+0000 7f30bb7e6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b415e8f0 0x7f30b4158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:39.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.490+0000 7f30bb7e6640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30b415f240 0x7f30b4158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:39.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.490+0000 7f30bb7e6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f30b40ca4d0 con 0x7f30b415e8f0 2024-09-17T03:15:39.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.490+0000 7f30bb7e6640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f30b40cc660 con 0x7f30b40af680 2024-09-17T03:15:39.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.490+0000 7f30bb7e6640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f30b40ce650 con 0x7f30b415f240 2024-09-17T03:15:39.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.490+0000 7f30b8d81640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b415e8f0 0x7f30b4158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:39.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.490+0000 7f30b9582640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30b40af680 0x7f30b40b5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:39.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.490+0000 7f30b8d81640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b415e8f0 0x7f30b4158920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36014/0 (socket says 172.21.15.87:36014) 2024-09-17T03:15:39.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.490+0000 7f30b8d81640 1 -- 172.21.15.87:0/390906003 learned_addr learned my addr 172.21.15.87:0/390906003 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:39.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.490+0000 7f30b9582640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30b40af680 0x7f30b40b5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3300/0 says I am v2:172.21.15.87:40464/0 (socket says 172.21.15.87:40464) 2024-09-17T03:15:39.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.490+0000 7f30b9d83640 1 --2- 172.21.15.87:0/390906003 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30b415f240 0x7f30b4158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:39.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.490+0000 7f30b8d81640 1 -- 172.21.15.87:0/390906003 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30b415f240 msgr2=0x7f30b4158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:39.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.490+0000 7f30b8d81640 1 --2- 172.21.15.87:0/390906003 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30b415f240 0x7f30b4158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:39.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.490+0000 7f30b8d81640 1 -- 172.21.15.87:0/390906003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30b40af680 msgr2=0x7f30b40b5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:39.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.490+0000 7f30b8d81640 1 --2- 172.21.15.87:0/390906003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30b40af680 0x7f30b40b5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:39.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.490+0000 7f30b8d81640 1 -- 172.21.15.87:0/390906003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b41f2920 con 0x7f30b415e8f0 2024-09-17T03:15:39.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.490+0000 7f30b9582640 1 --2- 172.21.15.87:0/390906003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30b40af680 0x7f30b40b5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-17T03:15:39.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.490+0000 7f30b8d81640 1 --2- 172.21.15.87:0/390906003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b415e8f0 0x7f30b4158920 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f30a4034040 tx=0x7f30a40038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:39.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.490+0000 7f30a9ffb640 1 -- 172.21.15.87:0/390906003 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a403e030 con 0x7f30b415e8f0 2024-09-17T03:15:39.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.490+0000 7f30a9ffb640 1 -- 172.21.15.87:0/390906003 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30a4003db0 con 0x7f30b415e8f0 2024-09-17T03:15:39.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.490+0000 7f30bb7e6640 1 -- 172.21.15.87:0/390906003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30b41f2c00 con 0x7f30b415e8f0 2024-09-17T03:15:39.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.490+0000 7f30a9ffb640 1 -- 172.21.15.87:0/390906003 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a4035040 con 0x7f30b415e8f0 2024-09-17T03:15:39.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.491+0000 7f30bb7e6640 1 -- 172.21.15.87:0/390906003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30b41f31c0 con 0x7f30b415e8f0 2024-09-17T03:15:39.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.492+0000 7f30bb7e6640 1 -- 172.21.15.87:0/390906003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30b40ceb70 con 0x7f30b415e8f0 2024-09-17T03:15:39.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.492+0000 7f30a9ffb640 1 -- 172.21.15.87:0/390906003 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f30a4041030 con 0x7f30b415e8f0 2024-09-17T03:15:39.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.493+0000 7f30a9ffb640 1 --2- 172.21.15.87:0/390906003 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3098078b90 0x7f309807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:39.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.493+0000 7f30a9ffb640 1 -- 172.21.15.87:0/390906003 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7f30a40bb420 con 0x7f30b415e8f0 2024-09-17T03:15:39.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.493+0000 7f30b9582640 1 --2- 172.21.15.87:0/390906003 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3098078b90 0x7f309807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:39.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.494+0000 7f30b9582640 1 --2- 172.21.15.87:0/390906003 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3098078b90 0x7f309807b0a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f30b00095f0 tx=0x7f30b0005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:39.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.497+0000 7f30a9ffb640 1 -- 172.21.15.87:0/390906003 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f30a4083180 con 0x7f30b415e8f0 2024-09-17T03:15:39.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.672+0000 7f30bb7e6640 1 -- 172.21.15.87:0/390906003 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f30b415ae80 con 0x7f30b415e8f0 2024-09-17T03:15:39.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.673+0000 7f30a9ffb640 1 -- 172.21.15.87:0/390906003 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+16706 (secure 0 0 0) 0x7f30a4087bb0 con 0x7f30b415e8f0 2024-09-17T03:15:39.672 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:39.672 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":49,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:15:38.707328+0000","last_up_change":"2024-09-17T03:14:01.833605+0000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:14:03.433404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:15:36.355536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-17T03:15:37.674416+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6825","nonce":2430409749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6827","nonce":2430409749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6831","nonce":2430409749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6829","nonce":2430409749}]},"public_addr":"172.21.15.87:6825/2430409749","cluster_addr":"172.21.15.87:6827/2430409749","heartbeat_back_addr":"172.21.15.87:6831/2430409749","heartbeat_front_addr":"172.21.15.87:6829/2430409749","state":["exists","up"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6809","nonce":3467605435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6811","nonce":3467605435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6815","nonce":3467605435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6813","nonce":3467605435}]},"public_addr":"172.21.15.87:6809/3467605435","cluster_addr":"172.21.15.87:6811/3467605435","heartbeat_back_addr":"172.21.15.87:6815/3467605435","heartbeat_front_addr":"172.21.15.87:6813/3467605435","state":["exists","up"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6801","nonce":1450834766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6803","nonce":1450834766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6807","nonce":1450834766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6805","nonce":1450834766}]},"public_addr":"172.21.15.87:6801/1450834766","cluster_addr":"172.21.15.87:6803/1450834766","heartbeat_back_addr":"172.21.15.87:6807/1450834766","heartbeat_front_addr":"172.21.15.87:6805/1450834766","state":["exists","up"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6817","nonce":4253161216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6819","nonce":4253161216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6823","nonce":4253161216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6821","nonce":4253161216}]},"public_addr":"172.21.15.87:6817/4253161216","cluster_addr":"172.21.15.87:6819/4253161216","heartbeat_back_addr":"172.21.15.87:6823/4253161216","heartbeat_front_addr":"172.21.15.87:6821/4253161216","state":["exists","up"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6803","nonce":2414907089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6805","nonce":2414907089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6809","nonce":2414907089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6807","nonce":2414907089}]},"public_addr":"172.21.15.149:6803/2414907089","cluster_addr":"172.21.15.149:6805/2414907089","heartbeat_back_addr":"172.21.15.149:6809/2414907089","heartbeat_front_addr":"172.21.15.149:6807/2414907089","state":["exists","up"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6827","nonce":2182999530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6829","nonce":2182999530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6833","nonce":2182999530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6831","nonce":2182999530}]},"public_addr":"172.21.15.149:6827/2182999530","cluster_addr":"172.21.15.149:6829/2182999530","heartbeat_back_addr":"172.21.15.149:6833/2182999530","heartbeat_front_addr":"172.21.15.149:6831/2182999530","state":["exists","up"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6811","nonce":4289519146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6813","nonce":4289519146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6817","nonce":4289519146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6815","nonce":4289519146}]},"public_addr":"172.21.15.149:6811/4289519146","cluster_addr":"172.21.15.149:6813/4289519146","heartbeat_back_addr":"172.21.15.149:6817/4289519146","heartbeat_front_addr":"172.21.15.149:6815/4289519146","state":["exists","up"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6819","nonce":2771010477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6821","nonce":2771010477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6825","nonce":2771010477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6823","nonce":2771010477}]},"public_addr":"172.21.15.149:6819/2771010477","cluster_addr":"172.21.15.149:6821/2771010477","heartbeat_back_addr":"172.21.15.149:6825/2771010477","heartbeat_front_addr":"172.21.15.149:6823/2771010477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.313297+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.642861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.654781+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.302606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.385791+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.915266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.121365+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.394432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:15:39.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.675+0000 7f30bb7e6640 1 -- 172.21.15.87:0/390906003 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3098078b90 msgr2=0x7f309807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:39.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.675+0000 7f30bb7e6640 1 --2- 172.21.15.87:0/390906003 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3098078b90 0x7f309807b0a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f30b00095f0 tx=0x7f30b0005f90 comp rx=0 tx=0).stop 2024-09-17T03:15:39.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.675+0000 7f30bb7e6640 1 -- 172.21.15.87:0/390906003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b415e8f0 msgr2=0x7f30b4158920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:39.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.675+0000 7f30bb7e6640 1 --2- 172.21.15.87:0/390906003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b415e8f0 0x7f30b4158920 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f30a4034040 tx=0x7f30a40038a0 comp rx=0 tx=0).stop 2024-09-17T03:15:39.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.675+0000 7f30bb7e6640 1 -- 172.21.15.87:0/390906003 shutdown_connections 2024-09-17T03:15:39.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.675+0000 7f30bb7e6640 1 --2- 172.21.15.87:0/390906003 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f3098078b90 0x7f309807b0a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:39.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.675+0000 7f30bb7e6640 1 --2- 172.21.15.87:0/390906003 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f30b415f240 0x7f30b4158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:39.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.675+0000 7f30bb7e6640 1 --2- 172.21.15.87:0/390906003 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30b415e8f0 0x7f30b4158920 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:39.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.675+0000 7f30bb7e6640 1 --2- 172.21.15.87:0/390906003 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f30b40af680 0x7f30b40b5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:39.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.675+0000 7f30bb7e6640 1 -- 172.21.15.87:0/390906003 >> 172.21.15.87:0/390906003 conn(0x7f30b406ee70 msgr2=0x7f30b40747c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:39.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.675+0000 7f30bb7e6640 1 -- 172.21.15.87:0/390906003 shutdown_connections 2024-09-17T03:15:39.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.675+0000 7f30bb7e6640 1 -- 172.21.15.87:0/390906003 wait complete. 2024-09-17T03:15:39.685 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:15:39.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.800+0000 7fbd62da8640 1 -- 172.21.15.87:0/246081843 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd48004e60 msgr2=0x7fbd480052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:39.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.800+0000 7fbd62da8640 1 --2- 172.21.15.87:0/246081843 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd48004e60 0x7fbd480052b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c074eb0 tx=0x7fbd48005830 comp rx=0 tx=0).stop 2024-09-17T03:15:39.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.800+0000 7fbd61345640 1 -- 172.21.15.87:0/246081843 reap_dead start 2024-09-17T03:15:39.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.800+0000 7fbd62da8640 1 -- 172.21.15.87:0/246081843 shutdown_connections 2024-09-17T03:15:39.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.800+0000 7fbd62da8640 1 -- 172.21.15.87:0/246081843 >> 172.21.15.87:0/246081843 conn(0x7fbd5c069360 msgr2=0x7fbd5c0ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:39.799 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.800+0000 7fbd62da8640 1 -- 172.21.15.87:0/246081843 shutdown_connections 2024-09-17T03:15:39.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.800+0000 7fbd62da8640 1 -- 172.21.15.87:0/246081843 wait complete. 2024-09-17T03:15:39.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.801+0000 7fbd62da8640 1 Processor -- start 2024-09-17T03:15:39.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.801+0000 7fbd62da8640 1 -- start start 2024-09-17T03:15:39.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.803+0000 7fbd62da8640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd48004e60 0x7fbd5c0cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:39.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.803+0000 7fbd62da8640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd5c0c6300 0x7fbd5c155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:39.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.803+0000 7fbd62da8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd5c0b5ba0 0x7fbd5c0b5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:39.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.803+0000 7fbd62da8640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c165530 con 0x7fbd5c0b5ba0 2024-09-17T03:15:39.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.803+0000 7fbd62da8640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbd5c1653b0 con 0x7fbd48004e60 2024-09-17T03:15:39.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.803+0000 7fbd62da8640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbd5c1656b0 con 0x7fbd5c0c6300 2024-09-17T03:15:39.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.803+0000 7fbd61345640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd5c0b5ba0 0x7fbd5c0b5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:39.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.803+0000 7fbd61345640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd5c0b5ba0 0x7fbd5c0b5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36038/0 (socket says 172.21.15.87:36038) 2024-09-17T03:15:39.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.803+0000 7fbd61345640 1 -- 172.21.15.87:0/952394823 learned_addr learned my addr 172.21.15.87:0/952394823 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:39.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.803+0000 7fbd60b44640 1 --2- 172.21.15.87:0/952394823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd48004e60 0x7fbd5c0cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:39.804 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.803+0000 7fbd5bfff640 1 --2- 172.21.15.87:0/952394823 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd5c0c6300 0x7fbd5c155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:39.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.803+0000 7fbd61345640 1 -- 172.21.15.87:0/952394823 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd5c0c6300 msgr2=0x7fbd5c155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:39.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.803+0000 7fbd61345640 1 --2- 172.21.15.87:0/952394823 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd5c0c6300 0x7fbd5c155270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:39.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.803+0000 7fbd61345640 1 -- 172.21.15.87:0/952394823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd48004e60 msgr2=0x7fbd5c0cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:39.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.803+0000 7fbd61345640 1 --2- 172.21.15.87:0/952394823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd48004e60 0x7fbd5c0cbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:39.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.803+0000 7fbd61345640 1 -- 172.21.15.87:0/952394823 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd5c0b64f0 con 0x7fbd5c0b5ba0 2024-09-17T03:15:39.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.803+0000 7fbd61345640 1 --2- 172.21.15.87:0/952394823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd5c0b5ba0 0x7fbd5c0b5fb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00bbe0 tx=0x7fbd4c032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:39.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.804+0000 7fbd597fa640 1 -- 172.21.15.87:0/952394823 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c00b6f0 con 0x7fbd5c0b5ba0 2024-09-17T03:15:39.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.804+0000 7fbd597fa640 1 -- 172.21.15.87:0/952394823 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd4c005310 con 0x7fbd5c0b5ba0 2024-09-17T03:15:39.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.804+0000 7fbd62da8640 1 -- 172.21.15.87:0/952394823 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd5c1f88f0 con 0x7fbd5c0b5ba0 2024-09-17T03:15:39.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.804+0000 7fbd597fa640 1 -- 172.21.15.87:0/952394823 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c0047a0 con 0x7fbd5c0b5ba0 2024-09-17T03:15:39.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.804+0000 7fbd62da8640 1 -- 172.21.15.87:0/952394823 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd5c1f8eb0 con 0x7fbd5c0b5ba0 2024-09-17T03:15:39.805 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.805+0000 7fbd62da8640 1 -- 172.21.15.87:0/952394823 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd5c0cc140 con 0x7fbd5c0b5ba0 2024-09-17T03:15:39.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.805+0000 7fbd597fa640 1 -- 172.21.15.87:0/952394823 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbd4c004940 con 0x7fbd5c0b5ba0 2024-09-17T03:15:39.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.806+0000 7fbd597fa640 1 --2- 172.21.15.87:0/952394823 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbd3c078b90 0x7fbd3c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:39.806 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.806+0000 7fbd597fa640 1 -- 172.21.15.87:0/952394823 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7fbd4c0bbcc0 con 0x7fbd5c0b5ba0 2024-09-17T03:15:39.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.806+0000 7fbd60b44640 1 --2- 172.21.15.87:0/952394823 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbd3c078b90 0x7fbd3c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:39.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.807+0000 7fbd60b44640 1 --2- 172.21.15.87:0/952394823 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbd3c078b90 0x7fbd3c07b0a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbd54000c00 tx=0x7fbd54006b70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:39.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.811+0000 7fbd597fa640 1 -- 172.21.15.87:0/952394823 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fbd5c0b64f0 con 0x7fbd5c0b5ba0 2024-09-17T03:15:39.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.986+0000 7fbd62da8640 1 -- 172.21.15.87:0/952394823 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd5c160c20 con 0x7fbd5c0b5ba0 2024-09-17T03:15:39.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.987+0000 7fbd597fa640 1 -- 172.21.15.87:0/952394823 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+16706 (secure 0 0 0) 0x7fbd4c083ab0 con 0x7fbd5c0b5ba0 2024-09-17T03:15:39.986 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:39.986 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":49,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:15:38.707328+0000","last_up_change":"2024-09-17T03:14:01.833605+0000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:14:03.433404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:15:36.355536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-17T03:15:37.674416+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6825","nonce":2430409749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6827","nonce":2430409749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6831","nonce":2430409749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6829","nonce":2430409749}]},"public_addr":"172.21.15.87:6825/2430409749","cluster_addr":"172.21.15.87:6827/2430409749","heartbeat_back_addr":"172.21.15.87:6831/2430409749","heartbeat_front_addr":"172.21.15.87:6829/2430409749","state":["exists","up"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6809","nonce":3467605435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6811","nonce":3467605435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6815","nonce":3467605435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6813","nonce":3467605435}]},"public_addr":"172.21.15.87:6809/3467605435","cluster_addr":"172.21.15.87:6811/3467605435","heartbeat_back_addr":"172.21.15.87:6815/3467605435","heartbeat_front_addr":"172.21.15.87:6813/3467605435","state":["exists","up"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6801","nonce":1450834766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6803","nonce":1450834766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6807","nonce":1450834766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6805","nonce":1450834766}]},"public_addr":"172.21.15.87:6801/1450834766","cluster_addr":"172.21.15.87:6803/1450834766","heartbeat_back_addr":"172.21.15.87:6807/1450834766","heartbeat_front_addr":"172.21.15.87:6805/1450834766","state":["exists","up"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6817","nonce":4253161216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6819","nonce":4253161216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6823","nonce":4253161216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6821","nonce":4253161216}]},"public_addr":"172.21.15.87:6817/4253161216","cluster_addr":"172.21.15.87:6819/4253161216","heartbeat_back_addr":"172.21.15.87:6823/4253161216","heartbeat_front_addr":"172.21.15.87:6821/4253161216","state":["exists","up"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6803","nonce":2414907089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6805","nonce":2414907089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6809","nonce":2414907089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6807","nonce":2414907089}]},"public_addr":"172.21.15.149:6803/2414907089","cluster_addr":"172.21.15.149:6805/2414907089","heartbeat_back_addr":"172.21.15.149:6809/2414907089","heartbeat_front_addr":"172.21.15.149:6807/2414907089","state":["exists","up"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6827","nonce":2182999530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6829","nonce":2182999530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6833","nonce":2182999530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6831","nonce":2182999530}]},"public_addr":"172.21.15.149:6827/2182999530","cluster_addr":"172.21.15.149:6829/2182999530","heartbeat_back_addr":"172.21.15.149:6833/2182999530","heartbeat_front_addr":"172.21.15.149:6831/2182999530","state":["exists","up"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6811","nonce":4289519146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6813","nonce":4289519146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6817","nonce":4289519146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6815","nonce":4289519146}]},"public_addr":"172.21.15.149:6811/4289519146","cluster_addr":"172.21.15.149:6813/4289519146","heartbeat_back_addr":"172.21.15.149:6817/4289519146","heartbeat_front_addr":"172.21.15.149:6815/4289519146","state":["exists","up"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6819","nonce":2771010477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6821","nonce":2771010477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6825","nonce":2771010477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6823","nonce":2771010477}]},"public_addr":"172.21.15.149:6819/2771010477","cluster_addr":"172.21.15.149:6821/2771010477","heartbeat_back_addr":"172.21.15.149:6825/2771010477","heartbeat_front_addr":"172.21.15.149:6823/2771010477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.313297+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.642861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.654781+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.302606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.385791+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.915266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.121365+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.394432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:15:39.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.989+0000 7fbd62da8640 1 -- 172.21.15.87:0/952394823 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbd3c078b90 msgr2=0x7fbd3c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:39.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.989+0000 7fbd62da8640 1 --2- 172.21.15.87:0/952394823 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbd3c078b90 0x7fbd3c07b0a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbd54000c00 tx=0x7fbd54006b70 comp rx=0 tx=0).stop 2024-09-17T03:15:39.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.989+0000 7fbd62da8640 1 -- 172.21.15.87:0/952394823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd5c0b5ba0 msgr2=0x7fbd5c0b5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:39.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.989+0000 7fbd62da8640 1 --2- 172.21.15.87:0/952394823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd5c0b5ba0 0x7fbd5c0b5fb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00bbe0 tx=0x7fbd4c032c30 comp rx=0 tx=0).stop 2024-09-17T03:15:39.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.989+0000 7fbd62da8640 1 -- 172.21.15.87:0/952394823 shutdown_connections 2024-09-17T03:15:39.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.989+0000 7fbd62da8640 1 --2- 172.21.15.87:0/952394823 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbd3c078b90 0x7fbd3c07b0a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:39.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.989+0000 7fbd62da8640 1 --2- 172.21.15.87:0/952394823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd5c0b5ba0 0x7fbd5c0b5fb0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:39.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.989+0000 7fbd62da8640 1 --2- 172.21.15.87:0/952394823 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbd5c0c6300 0x7fbd5c155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:39.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.989+0000 7fbd62da8640 1 --2- 172.21.15.87:0/952394823 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbd48004e60 0x7fbd5c0cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:39.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.989+0000 7fbd62da8640 1 -- 172.21.15.87:0/952394823 >> 172.21.15.87:0/952394823 conn(0x7fbd5c069360 msgr2=0x7fbd5c0d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:39.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.989+0000 7fbd62da8640 1 -- 172.21.15.87:0/952394823 shutdown_connections 2024-09-17T03:15:39.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:39.989+0000 7fbd62da8640 1 -- 172.21.15.87:0/952394823 wait complete. 2024-09-17T03:15:39.999 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-17T03:15:40.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.115+0000 7f56a4bc3640 1 -- 172.21.15.87:0/1238870408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5688004d40 msgr2=0x7f5688005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:40.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.115+0000 7f56a4bc3640 1 --2- 172.21.15.87:0/1238870408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5688004d40 0x7f5688005150 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f56a00b70c0 tx=0x7f5688005830 comp rx=0 tx=0).stop 2024-09-17T03:15:40.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.115+0000 7f569ed9d640 1 -- 172.21.15.87:0/1238870408 reap_dead start 2024-09-17T03:15:40.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.115+0000 7f56a4bc3640 1 -- 172.21.15.87:0/1238870408 shutdown_connections 2024-09-17T03:15:40.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.115+0000 7f56a4bc3640 1 -- 172.21.15.87:0/1238870408 >> 172.21.15.87:0/1238870408 conn(0x7f56a009d840 msgr2=0x7f56a008bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:40.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.115+0000 7f56a4bc3640 1 -- 172.21.15.87:0/1238870408 shutdown_connections 2024-09-17T03:15:40.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.115+0000 7f56a4bc3640 1 -- 172.21.15.87:0/1238870408 wait complete. 2024-09-17T03:15:40.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.117+0000 7f56a4bc3640 1 Processor -- start 2024-09-17T03:15:40.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.117+0000 7f56a4bc3640 1 -- start start 2024-09-17T03:15:40.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.118+0000 7f56a4bc3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5688004d40 0x7f56a015b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:40.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.119+0000 7f56a4bc3640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56a01f5aa0 0x7f56a00b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:40.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.119+0000 7f56a4bc3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56a0158fd0 0x7f56a00b4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:40.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.119+0000 7f56a4bc3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f56a00d3360 con 0x7f5688004d40 2024-09-17T03:15:40.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.119+0000 7f56a4bc3640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f56a00d31e0 con 0x7f56a0158fd0 2024-09-17T03:15:40.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.119+0000 7f56a4bc3640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f56a00cefa0 con 0x7f56a01f5aa0 2024-09-17T03:15:40.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.119+0000 7f569e59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5688004d40 0x7f56a015b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:40.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.119+0000 7f569e59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5688004d40 0x7f56a015b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36054/0 (socket says 172.21.15.87:36054) 2024-09-17T03:15:40.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.119+0000 7f569e59c640 1 -- 172.21.15.87:0/3184876587 learned_addr learned my addr 172.21.15.87:0/3184876587 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:40.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.119+0000 7f569ed9d640 1 --2- 172.21.15.87:0/3184876587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56a0158fd0 0x7f56a00b4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:40.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.119+0000 7f569dd9b640 1 --2- 172.21.15.87:0/3184876587 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56a01f5aa0 0x7f56a00b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:40.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.119+0000 7f569e59c640 1 -- 172.21.15.87:0/3184876587 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56a01f5aa0 msgr2=0x7f56a00b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:40.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.119+0000 7f569e59c640 1 --2- 172.21.15.87:0/3184876587 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56a01f5aa0 0x7f56a00b4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:40.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.119+0000 7f569e59c640 1 -- 172.21.15.87:0/3184876587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56a0158fd0 msgr2=0x7f56a00b4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:40.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.119+0000 7f569e59c640 1 --2- 172.21.15.87:0/3184876587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56a0158fd0 0x7f56a00b4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:40.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.119+0000 7f569e59c640 1 -- 172.21.15.87:0/3184876587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a00b5240 con 0x7f5688004d40 2024-09-17T03:15:40.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.119+0000 7f569ed9d640 1 --2- 172.21.15.87:0/3184876587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56a0158fd0 0x7f56a00b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:15:40.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.119+0000 7f569e59c640 1 --2- 172.21.15.87:0/3184876587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5688004d40 0x7f56a015b3a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5690011d70 tx=0x7f569000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:40.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.119+0000 7f568effd640 1 -- 172.21.15.87:0/3184876587 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690003b80 con 0x7f5688004d40 2024-09-17T03:15:40.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.119+0000 7f568effd640 1 -- 172.21.15.87:0/3184876587 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5690003d20 con 0x7f5688004d40 2024-09-17T03:15:40.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.119+0000 7f56a4bc3640 1 -- 172.21.15.87:0/3184876587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a01f6280 con 0x7f5688004d40 2024-09-17T03:15:40.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.119+0000 7f568effd640 1 -- 172.21.15.87:0/3184876587 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690004d60 con 0x7f5688004d40 2024-09-17T03:15:40.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.119+0000 7f56a4bc3640 1 -- 172.21.15.87:0/3184876587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a01f6840 con 0x7f5688004d40 2024-09-17T03:15:40.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.120+0000 7f56a4bc3640 1 -- 172.21.15.87:0/3184876587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a00b3660 con 0x7f5688004d40 2024-09-17T03:15:40.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.121+0000 7f568effd640 1 -- 172.21.15.87:0/3184876587 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5690004030 con 0x7f5688004d40 2024-09-17T03:15:40.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.122+0000 7f568effd640 1 --2- 172.21.15.87:0/3184876587 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5680078b90 0x7f568007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:40.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.122+0000 7f568effd640 1 -- 172.21.15.87:0/3184876587 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5709+0+0 (secure 0 0 0) 0x7f569009bd30 con 0x7f5688004d40 2024-09-17T03:15:40.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.122+0000 7f569dd9b640 1 --2- 172.21.15.87:0/3184876587 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5680078b90 0x7f568007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:40.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.122+0000 7f569dd9b640 1 --2- 172.21.15.87:0/3184876587 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5680078b90 0x7f568007b0a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f56a01f8ff0 tx=0x7f5694005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:40.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.126+0000 7f568effd640 1 -- 172.21.15.87:0/3184876587 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f5690064210 con 0x7f5688004d40 2024-09-17T03:15:40.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.336+0000 7f56a4bc3640 1 -- 172.21.15.87:0/3184876587 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f56a0158db0 con 0x7f5688004d40 2024-09-17T03:15:40.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.360+0000 7f568effd640 1 -- 172.21.15.87:0/3184876587 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v19) ==== 118+0+0 (secure 0 0 0) 0x7f5690063b10 con 0x7f5688004d40 2024-09-17T03:15:40.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.363+0000 7f56a4bc3640 1 -- 172.21.15.87:0/3184876587 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5680078b90 msgr2=0x7f568007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:40.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.363+0000 7f56a4bc3640 1 --2- 172.21.15.87:0/3184876587 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5680078b90 0x7f568007b0a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f56a01f8ff0 tx=0x7f5694005f90 comp rx=0 tx=0).stop 2024-09-17T03:15:40.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.363+0000 7f56a4bc3640 1 -- 172.21.15.87:0/3184876587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5688004d40 msgr2=0x7f56a015b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:40.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.363+0000 7f56a4bc3640 1 --2- 172.21.15.87:0/3184876587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5688004d40 0x7f56a015b3a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5690011d70 tx=0x7f569000f790 comp rx=0 tx=0).stop 2024-09-17T03:15:40.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.363+0000 7f56a4bc3640 1 -- 172.21.15.87:0/3184876587 shutdown_connections 2024-09-17T03:15:40.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.363+0000 7f56a4bc3640 1 --2- 172.21.15.87:0/3184876587 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5680078b90 0x7f568007b0a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:40.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.363+0000 7f56a4bc3640 1 --2- 172.21.15.87:0/3184876587 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f56a0158fd0 0x7f56a00b4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:40.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.363+0000 7f56a4bc3640 1 --2- 172.21.15.87:0/3184876587 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f56a01f5aa0 0x7f56a00b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:40.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.363+0000 7f56a4bc3640 1 --2- 172.21.15.87:0/3184876587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5688004d40 0x7f56a015b3a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:40.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.363+0000 7f56a4bc3640 1 -- 172.21.15.87:0/3184876587 >> 172.21.15.87:0/3184876587 conn(0x7f56a009d840 msgr2=0x7f56a00d44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:40.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.363+0000 7f56a4bc3640 1 -- 172.21.15.87:0/3184876587 shutdown_connections 2024-09-17T03:15:40.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.363+0000 7f56a4bc3640 1 -- 172.21.15.87:0/3184876587 wait complete. 2024-09-17T03:15:40.373 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-17T03:15:40.373 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-17T03:15:40.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.489+0000 7fbb85df1640 1 -- 172.21.15.87:0/3060492112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb800b3d00 msgr2=0x7fbb800b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:40.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.489+0000 7fbb85df1640 1 --2- 172.21.15.87:0/3060492112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb800b3d00 0x7fbb800b4110 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fbb800b8370 tx=0x7fbb7802a980 comp rx=0 tx=0).stop 2024-09-17T03:15:40.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.489+0000 7fbb85df1640 1 -- 172.21.15.87:0/3060492112 shutdown_connections 2024-09-17T03:15:40.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.489+0000 7fbb85df1640 1 --2- 172.21.15.87:0/3060492112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb800b3d00 0x7fbb800b4110 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:40.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.489+0000 7fbb85df1640 1 --2- 172.21.15.87:0/3060492112 >> v2:172.21.15.149:3300/0 conn(0x7fbb80160150 0x7fbb80160560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:40.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.489+0000 7fbb85df1640 1 -- 172.21.15.87:0/3060492112 >> 172.21.15.87:0/3060492112 conn(0x7fbb80069330 msgr2=0x7fbb800ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:40.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.489+0000 7fbb85df1640 1 -- 172.21.15.87:0/3060492112 shutdown_connections 2024-09-17T03:15:40.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.489+0000 7fbb85df1640 1 -- 172.21.15.87:0/3060492112 wait complete. 2024-09-17T03:15:40.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.491+0000 7fbb85df1640 1 Processor -- start 2024-09-17T03:15:40.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.491+0000 7fbb85df1640 1 -- start start 2024-09-17T03:15:40.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.492+0000 7fbb85df1640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbb800b3d00 0x7fbb8014c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:40.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.492+0000 7fbb85df1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb8015d5e0 0x7fbb8015ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:40.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.492+0000 7fbb85df1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb80160150 0x7fbb801541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:40.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.492+0000 7fbb85df1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbb800d0300 con 0x7fbb8015d5e0 2024-09-17T03:15:40.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.492+0000 7fbb85df1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbb800d0180 con 0x7fbb80160150 2024-09-17T03:15:40.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.492+0000 7fbb85df1640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbb800d0480 con 0x7fbb800b3d00 2024-09-17T03:15:40.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.492+0000 7fbb7effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb8015d5e0 0x7fbb8015ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:40.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.492+0000 7fbb7effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb8015d5e0 0x7fbb8015ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36066/0 (socket says 172.21.15.87:36066) 2024-09-17T03:15:40.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.492+0000 7fbb7effd640 1 -- 172.21.15.87:0/3082437414 learned_addr learned my addr 172.21.15.87:0/3082437414 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:40.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.493+0000 7fbb7ffff640 1 --2- 172.21.15.87:0/3082437414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb80160150 0x7fbb801541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:40.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.493+0000 7fbb7f7fe640 1 --2- 172.21.15.87:0/3082437414 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbb800b3d00 0x7fbb8014c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:40.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.493+0000 7fbb7effd640 1 -- 172.21.15.87:0/3082437414 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbb800b3d00 msgr2=0x7fbb8014c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:40.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.493+0000 7fbb7effd640 1 --2- 172.21.15.87:0/3082437414 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbb800b3d00 0x7fbb8014c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:40.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.493+0000 7fbb7effd640 1 -- 172.21.15.87:0/3082437414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb80160150 msgr2=0x7fbb801541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:40.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.493+0000 7fbb7effd640 1 --2- 172.21.15.87:0/3082437414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb80160150 0x7fbb801541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:40.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.493+0000 7fbb7effd640 1 -- 172.21.15.87:0/3082437414 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb8014c930 con 0x7fbb8015d5e0 2024-09-17T03:15:40.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.493+0000 7fbb7effd640 1 --2- 172.21.15.87:0/3082437414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb8015d5e0 0x7fbb8015ef50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbb7000d700 tx=0x7fbb7000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:40.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.493+0000 7fbb77fff640 1 -- 172.21.15.87:0/3082437414 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70017070 con 0x7fbb8015d5e0 2024-09-17T03:15:40.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.493+0000 7fbb77fff640 1 -- 172.21.15.87:0/3082437414 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb70005310 con 0x7fbb8015d5e0 2024-09-17T03:15:40.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.493+0000 7fbb85df1640 1 -- 172.21.15.87:0/3082437414 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb80157d30 con 0x7fbb8015d5e0 2024-09-17T03:15:40.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.493+0000 7fbb77fff640 1 -- 172.21.15.87:0/3082437414 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70009820 con 0x7fbb8015d5e0 2024-09-17T03:15:40.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.493+0000 7fbb85df1640 1 -- 172.21.15.87:0/3082437414 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb8015e750 con 0x7fbb8015d5e0 2024-09-17T03:15:40.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.494+0000 7fbb85df1640 1 -- 172.21.15.87:0/3082437414 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb80158250 con 0x7fbb8015d5e0 2024-09-17T03:15:40.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.495+0000 7fbb77fff640 1 -- 172.21.15.87:0/3082437414 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbb700099c0 con 0x7fbb8015d5e0 2024-09-17T03:15:40.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.495+0000 7fbb77fff640 1 --2- 172.21.15.87:0/3082437414 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbb60078b90 0x7fbb6007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:40.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.496+0000 7fbb77fff640 1 -- 172.21.15.87:0/3082437414 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fbb7009adb0 con 0x7fbb8015d5e0 2024-09-17T03:15:40.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.496+0000 7fbb7f7fe640 1 --2- 172.21.15.87:0/3082437414 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbb60078b90 0x7fbb6007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:40.495 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.496+0000 7fbb7f7fe640 1 --2- 172.21.15.87:0/3082437414 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbb60078b90 0x7fbb6007b0a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbb68006fd0 tx=0x7fbb68009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:40.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.500+0000 7fbb77fff640 1 -- 172.21.15.87:0/3082437414 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fbb70062ba0 con 0x7fbb8015d5e0 2024-09-17T03:15:40.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:40.711+0000 7fbb85df1640 1 -- 172.21.15.87:0/3082437414 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fbb800d1b60 con 0x7fbb8015d5e0 2024-09-17T03:15:41.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.366+0000 7fbb77fff640 1 -- 172.21.15.87:0/3082437414 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7fbb700675d0 con 0x7fbb8015d5e0 2024-09-17T03:15:41.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.369+0000 7fbb85df1640 1 -- 172.21.15.87:0/3082437414 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbb60078b90 msgr2=0x7fbb6007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:41.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.369+0000 7fbb85df1640 1 --2- 172.21.15.87:0/3082437414 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbb60078b90 0x7fbb6007b0a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbb68006fd0 tx=0x7fbb68009060 comp rx=0 tx=0).stop 2024-09-17T03:15:41.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.369+0000 7fbb85df1640 1 -- 172.21.15.87:0/3082437414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb8015d5e0 msgr2=0x7fbb8015ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:41.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.369+0000 7fbb85df1640 1 --2- 172.21.15.87:0/3082437414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb8015d5e0 0x7fbb8015ef50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fbb7000d700 tx=0x7fbb7000dbc0 comp rx=0 tx=0).stop 2024-09-17T03:15:41.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.369+0000 7fbb85df1640 1 -- 172.21.15.87:0/3082437414 shutdown_connections 2024-09-17T03:15:41.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.369+0000 7fbb85df1640 1 --2- 172.21.15.87:0/3082437414 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbb60078b90 0x7fbb6007b0a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:41.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.369+0000 7fbb85df1640 1 --2- 172.21.15.87:0/3082437414 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbb80160150 0x7fbb801541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:41.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.369+0000 7fbb85df1640 1 --2- 172.21.15.87:0/3082437414 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb8015d5e0 0x7fbb8015ef50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:41.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.369+0000 7fbb85df1640 1 --2- 172.21.15.87:0/3082437414 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbb800b3d00 0x7fbb8014c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:41.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.369+0000 7fbb85df1640 1 -- 172.21.15.87:0/3082437414 >> 172.21.15.87:0/3082437414 conn(0x7fbb80069330 msgr2=0x7fbb800b2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:41.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.369+0000 7fbb85df1640 1 -- 172.21.15.87:0/3082437414 shutdown_connections 2024-09-17T03:15:41.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.369+0000 7fbb85df1640 1 -- 172.21.15.87:0/3082437414 wait complete. 2024-09-17T03:15:41.380 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-17T03:15:41.381 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:15:41.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.498+0000 7f0004ba1640 1 -- 172.21.15.87:0/1948951024 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00000b3d00 msgr2=0x7f00000b4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:41.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.498+0000 7f0004ba1640 1 --2- 172.21.15.87:0/1948951024 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00000b3d00 0x7f00000b4110 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7effe8005fd0 tx=0x7effe802a540 comp rx=0 tx=0).stop 2024-09-17T03:15:41.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.498+0000 7f0004ba1640 1 -- 172.21.15.87:0/1948951024 shutdown_connections 2024-09-17T03:15:41.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.498+0000 7f0004ba1640 1 --2- 172.21.15.87:0/1948951024 >> v2:172.21.15.149:3300/0 conn(0x7f00000d0460 0x7f00000d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:41.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.498+0000 7f0004ba1640 1 --2- 172.21.15.87:0/1948951024 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00000b3d00 0x7f00000b4110 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:41.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.498+0000 7f0004ba1640 1 -- 172.21.15.87:0/1948951024 >> 172.21.15.87:0/1948951024 conn(0x7f0000069360 msgr2=0x7f00000ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:41.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.498+0000 7f0004ba1640 1 -- 172.21.15.87:0/1948951024 shutdown_connections 2024-09-17T03:15:41.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.498+0000 7f0004ba1640 1 -- 172.21.15.87:0/1948951024 wait complete. 2024-09-17T03:15:41.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.500+0000 7f0004ba1640 1 Processor -- start 2024-09-17T03:15:41.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.500+0000 7f0004ba1640 1 -- start start 2024-09-17T03:15:41.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.502+0000 7f0004ba1640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00000b3d00 0x7f00001f63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:41.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.502+0000 7f0004ba1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00000d0460 0x7f00000c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:41.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.502+0000 7f0004ba1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f000015c6c0 0x7f00000c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:41.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.502+0000 7f0004ba1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f00000d0f30 con 0x7f000015c6c0 2024-09-17T03:15:41.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.502+0000 7f0004ba1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f00000d0db0 con 0x7f00000d0460 2024-09-17T03:15:41.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.502+0000 7f0004ba1640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f00000d10b0 con 0x7f00000b3d00 2024-09-17T03:15:41.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.502+0000 7efffed9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f000015c6c0 0x7f00000c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:41.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.502+0000 7efffed9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f000015c6c0 0x7f00000c7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36068/0 (socket says 172.21.15.87:36068) 2024-09-17T03:15:41.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.502+0000 7efffed9d640 1 -- 172.21.15.87:0/917875230 learned_addr learned my addr 172.21.15.87:0/917875230 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:41.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.502+0000 7efffdd9b640 1 --2- 172.21.15.87:0/917875230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00000d0460 0x7f00000c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:41.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.502+0000 7efffe59c640 1 --2- 172.21.15.87:0/917875230 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00000b3d00 0x7f00001f63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:41.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.502+0000 7efffed9d640 1 -- 172.21.15.87:0/917875230 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00000b3d00 msgr2=0x7f00001f63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:41.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.502+0000 7efffed9d640 1 --2- 172.21.15.87:0/917875230 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00000b3d00 0x7f00001f63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:41.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.502+0000 7efffed9d640 1 -- 172.21.15.87:0/917875230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00000d0460 msgr2=0x7f00000c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:41.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.502+0000 7efffed9d640 1 --2- 172.21.15.87:0/917875230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00000d0460 0x7f00000c70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:41.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.502+0000 7efffed9d640 1 -- 172.21.15.87:0/917875230 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00001f3840 con 0x7f000015c6c0 2024-09-17T03:15:41.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.502+0000 7efffed9d640 1 --2- 172.21.15.87:0/917875230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f000015c6c0 0x7f00000c7630 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7efff000e940 tx=0x7efff000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:41.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.502+0000 7efff6ffd640 1 -- 172.21.15.87:0/917875230 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff000ccc0 con 0x7f000015c6c0 2024-09-17T03:15:41.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.502+0000 7efff6ffd640 1 -- 172.21.15.87:0/917875230 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efff0010430 con 0x7f000015c6c0 2024-09-17T03:15:41.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.502+0000 7f0004ba1640 1 -- 172.21.15.87:0/917875230 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00001f3b20 con 0x7f000015c6c0 2024-09-17T03:15:41.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.502+0000 7efff6ffd640 1 -- 172.21.15.87:0/917875230 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0010880 con 0x7f000015c6c0 2024-09-17T03:15:41.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.503+0000 7f0004ba1640 1 -- 172.21.15.87:0/917875230 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00001f40e0 con 0x7f000015c6c0 2024-09-17T03:15:41.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.503+0000 7f0004ba1640 1 -- 172.21.15.87:0/917875230 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00000cf620 con 0x7f000015c6c0 2024-09-17T03:15:41.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.504+0000 7efff6ffd640 1 -- 172.21.15.87:0/917875230 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efff001f020 con 0x7f000015c6c0 2024-09-17T03:15:41.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.504+0000 7efff6ffd640 1 --2- 172.21.15.87:0/917875230 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7effe0078bc0 0x7effe007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:41.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.505+0000 7efff6ffd640 1 -- 172.21.15.87:0/917875230 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7efff009b400 con 0x7f000015c6c0 2024-09-17T03:15:41.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.505+0000 7efffe59c640 1 --2- 172.21.15.87:0/917875230 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7effe0078bc0 0x7effe007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:41.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.505+0000 7efffe59c640 1 --2- 172.21.15.87:0/917875230 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7effe0078bc0 0x7effe007b0d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efff8001e20 tx=0x7efff8009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:41.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.509+0000 7efff6ffd640 1 -- 172.21.15.87:0/917875230 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f00000cf620 con 0x7f000015c6c0 2024-09-17T03:15:41.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.716+0000 7f0004ba1640 1 -- 172.21.15.87:0/917875230 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f00001f35c0 con 0x7f000015c6c0 2024-09-17T03:15:41.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.717+0000 7efff6ffd640 1 -- 172.21.15.87:0/917875230 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5229 (secure 0 0 0) 0x7f00001f35c0 con 0x7f000015c6c0 2024-09-17T03:15:41.716 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:41.716 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":20,"btime":"2024-09-17T03:15:41:361821+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5104,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/1606650259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1606650259},{"type":"v1","addr":"172.21.15.87:6834","nonce":1606650259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":5116,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6836/3876173555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3876173555},{"type":"v1","addr":"172.21.15.149:6836","nonce":3876173555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1657329632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":1657329632},{"type":"v1","addr":"172.21.15.87:6835","nonce":1657329632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:15:38.707286+0000","modified":"2024-09-17T03:15:40.712230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5487":{"gid":5487,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6837/3562056427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6835","nonce":3562056427},{"type":"v1","addr":"172.21.15.149:6837","nonce":3562056427}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5487,"qdb_cluster":[5487]},"id":3}]} 2024-09-17T03:15:41.716 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 20 2024-09-17T03:15:41.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.719+0000 7f0004ba1640 1 -- 172.21.15.87:0/917875230 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7effe0078bc0 msgr2=0x7effe007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:41.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.719+0000 7f0004ba1640 1 --2- 172.21.15.87:0/917875230 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7effe0078bc0 0x7effe007b0d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7efff8001e20 tx=0x7efff8009040 comp rx=0 tx=0).stop 2024-09-17T03:15:41.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.719+0000 7f0004ba1640 1 -- 172.21.15.87:0/917875230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f000015c6c0 msgr2=0x7f00000c7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:41.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.719+0000 7f0004ba1640 1 --2- 172.21.15.87:0/917875230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f000015c6c0 0x7f00000c7630 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7efff000e940 tx=0x7efff000ee00 comp rx=0 tx=0).stop 2024-09-17T03:15:41.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.719+0000 7f0004ba1640 1 -- 172.21.15.87:0/917875230 shutdown_connections 2024-09-17T03:15:41.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.719+0000 7f0004ba1640 1 --2- 172.21.15.87:0/917875230 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7effe0078bc0 0x7effe007b0d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:41.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.719+0000 7f0004ba1640 1 --2- 172.21.15.87:0/917875230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f000015c6c0 0x7f00000c7630 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:41.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.719+0000 7f0004ba1640 1 --2- 172.21.15.87:0/917875230 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f00000d0460 0x7f00000c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:41.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.719+0000 7f0004ba1640 1 --2- 172.21.15.87:0/917875230 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f00000b3d00 0x7f00001f63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:41.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.719+0000 7f0004ba1640 1 -- 172.21.15.87:0/917875230 >> 172.21.15.87:0/917875230 conn(0x7f0000069360 msgr2=0x7f000015b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:41.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.719+0000 7f0004ba1640 1 -- 172.21.15.87:0/917875230 shutdown_connections 2024-09-17T03:15:41.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.719+0000 7f0004ba1640 1 -- 172.21.15.87:0/917875230 wait complete. 2024-09-17T03:15:41.729 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:15:41.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.843+0000 7fe53a5c3640 1 -- 172.21.15.87:0/397341709 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe53415c6c0 msgr2=0x7fe534160c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:41.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.843+0000 7fe53a5c3640 1 --2- 172.21.15.87:0/397341709 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe53415c6c0 0x7fe534160c20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe524009860 tx=0x7fe52402a510 comp rx=0 tx=0).stop 2024-09-17T03:15:41.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.843+0000 7fe53a5c3640 1 -- 172.21.15.87:0/397341709 shutdown_connections 2024-09-17T03:15:41.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.843+0000 7fe53a5c3640 1 --2- 172.21.15.87:0/397341709 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe53415c6c0 0x7fe534160c20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:41.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.843+0000 7fe53a5c3640 1 -- 172.21.15.87:0/397341709 >> 172.21.15.87:0/397341709 conn(0x7fe534069360 msgr2=0x7fe5340ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:41.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.843+0000 7fe53a5c3640 1 -- 172.21.15.87:0/397341709 shutdown_connections 2024-09-17T03:15:41.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.843+0000 7fe53a5c3640 1 -- 172.21.15.87:0/397341709 wait complete. 2024-09-17T03:15:41.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.845+0000 7fe53a5c3640 1 Processor -- start 2024-09-17T03:15:41.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.845+0000 7fe53a5c3640 1 -- start start 2024-09-17T03:15:41.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.846+0000 7fe53a5c3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5340b3d00 0x7fe534151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:41.846 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.846+0000 7fe53a5c3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5340d0460 0x7fe5341f6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:41.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.846+0000 7fe53a5c3640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe53415c6c0 0x7fe5340ab050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:41.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.846+0000 7fe53a5c3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe5340d1100 con 0x7fe5340d0460 2024-09-17T03:15:41.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.846+0000 7fe53a5c3640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fe5340d0f80 con 0x7fe5340b3d00 2024-09-17T03:15:41.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.846+0000 7fe53a5c3640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fe5340d1280 con 0x7fe53415c6c0 2024-09-17T03:15:41.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.846+0000 7fe5337fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5340d0460 0x7fe5341f6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:41.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.846+0000 7fe5337fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5340d0460 0x7fe5341f6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36080/0 (socket says 172.21.15.87:36080) 2024-09-17T03:15:41.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.846+0000 7fe5337fe640 1 -- 172.21.15.87:0/3305534593 learned_addr learned my addr 172.21.15.87:0/3305534593 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:41.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.846+0000 7fe533fff640 1 --2- 172.21.15.87:0/3305534593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5340b3d00 0x7fe534151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:41.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.846+0000 7fe538b60640 1 --2- 172.21.15.87:0/3305534593 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe53415c6c0 0x7fe5340ab050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:41.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.847+0000 7fe5337fe640 1 -- 172.21.15.87:0/3305534593 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe53415c6c0 msgr2=0x7fe5340ab050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:41.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.847+0000 7fe5337fe640 1 --2- 172.21.15.87:0/3305534593 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe53415c6c0 0x7fe5340ab050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:41.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.847+0000 7fe5337fe640 1 -- 172.21.15.87:0/3305534593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5340b3d00 msgr2=0x7fe534151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:41.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.847+0000 7fe5337fe640 1 --2- 172.21.15.87:0/3305534593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5340b3d00 0x7fe534151880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:41.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.847+0000 7fe5337fe640 1 -- 172.21.15.87:0/3305534593 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5341f7660 con 0x7fe5340d0460 2024-09-17T03:15:41.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.847+0000 7fe533fff640 1 --2- 172.21.15.87:0/3305534593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5340b3d00 0x7fe534151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:41.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.847+0000 7fe5337fe640 1 --2- 172.21.15.87:0/3305534593 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5340d0460 0x7fe5341f6c00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe528001eb0 tx=0x7fe52800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:41.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.847+0000 7fe530ff9640 1 -- 172.21.15.87:0/3305534593 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528017070 con 0x7fe5340d0460 2024-09-17T03:15:41.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.847+0000 7fe530ff9640 1 -- 172.21.15.87:0/3305534593 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5280053c0 con 0x7fe5340d0460 2024-09-17T03:15:41.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.847+0000 7fe53a5c3640 1 -- 172.21.15.87:0/3305534593 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5341f7940 con 0x7fe5340d0460 2024-09-17T03:15:41.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.847+0000 7fe530ff9640 1 -- 172.21.15.87:0/3305534593 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528003830 con 0x7fe5340d0460 2024-09-17T03:15:41.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.847+0000 7fe53a5c3640 1 -- 172.21.15.87:0/3305534593 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe53415d100 con 0x7fe5340d0460 2024-09-17T03:15:41.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.848+0000 7fe53a5c3640 1 -- 172.21.15.87:0/3305534593 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe534160c20 con 0x7fe5340d0460 2024-09-17T03:15:41.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.849+0000 7fe530ff9640 1 -- 172.21.15.87:0/3305534593 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe52801a070 con 0x7fe5340d0460 2024-09-17T03:15:41.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.849+0000 7fe530ff9640 1 --2- 172.21.15.87:0/3305534593 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe520078b90 0x7fe52007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:41.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.849+0000 7fe530ff9640 1 -- 172.21.15.87:0/3305534593 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fe52809a4e0 con 0x7fe5340d0460 2024-09-17T03:15:41.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.850+0000 7fe533fff640 1 --2- 172.21.15.87:0/3305534593 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe520078b90 0x7fe52007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:41.849 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.850+0000 7fe533fff640 1 --2- 172.21.15.87:0/3305534593 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe520078b90 0x7fe52007b0a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe524034000 tx=0x7fe524004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:41.851 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:41.853+0000 7fe530ff9640 1 -- 172.21.15.87:0/3305534593 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fe528021020 con 0x7fe5340d0460 2024-09-17T03:15:42.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.031+0000 7fe53a5c3640 1 -- 172.21.15.87:0/3305534593 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe5341f7b10 con 0x7fe5340d0460 2024-09-17T03:15:42.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.032+0000 7fe530ff9640 1 -- 172.21.15.87:0/3305534593 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+16693 (secure 0 0 0) 0x7fe5280622d0 con 0x7fe5340d0460 2024-09-17T03:15:42.031 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:42.031 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":50,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:15:40.345194+0000","last_up_change":"2024-09-17T03:14:01.833605+0000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:14:03.433404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:15:36.355536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-17T03:15:37.674416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6825","nonce":2430409749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6827","nonce":2430409749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6831","nonce":2430409749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6829","nonce":2430409749}]},"public_addr":"172.21.15.87:6825/2430409749","cluster_addr":"172.21.15.87:6827/2430409749","heartbeat_back_addr":"172.21.15.87:6831/2430409749","heartbeat_front_addr":"172.21.15.87:6829/2430409749","state":["exists","up"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6809","nonce":3467605435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6811","nonce":3467605435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6815","nonce":3467605435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6813","nonce":3467605435}]},"public_addr":"172.21.15.87:6809/3467605435","cluster_addr":"172.21.15.87:6811/3467605435","heartbeat_back_addr":"172.21.15.87:6815/3467605435","heartbeat_front_addr":"172.21.15.87:6813/3467605435","state":["exists","up"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6801","nonce":1450834766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6803","nonce":1450834766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6807","nonce":1450834766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6805","nonce":1450834766}]},"public_addr":"172.21.15.87:6801/1450834766","cluster_addr":"172.21.15.87:6803/1450834766","heartbeat_back_addr":"172.21.15.87:6807/1450834766","heartbeat_front_addr":"172.21.15.87:6805/1450834766","state":["exists","up"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6817","nonce":4253161216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6819","nonce":4253161216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6823","nonce":4253161216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6821","nonce":4253161216}]},"public_addr":"172.21.15.87:6817/4253161216","cluster_addr":"172.21.15.87:6819/4253161216","heartbeat_back_addr":"172.21.15.87:6823/4253161216","heartbeat_front_addr":"172.21.15.87:6821/4253161216","state":["exists","up"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6803","nonce":2414907089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6805","nonce":2414907089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6809","nonce":2414907089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6807","nonce":2414907089}]},"public_addr":"172.21.15.149:6803/2414907089","cluster_addr":"172.21.15.149:6805/2414907089","heartbeat_back_addr":"172.21.15.149:6809/2414907089","heartbeat_front_addr":"172.21.15.149:6807/2414907089","state":["exists","up"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6827","nonce":2182999530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6829","nonce":2182999530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6833","nonce":2182999530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6831","nonce":2182999530}]},"public_addr":"172.21.15.149:6827/2182999530","cluster_addr":"172.21.15.149:6829/2182999530","heartbeat_back_addr":"172.21.15.149:6833/2182999530","heartbeat_front_addr":"172.21.15.149:6831/2182999530","state":["exists","up"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6811","nonce":4289519146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6813","nonce":4289519146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6817","nonce":4289519146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6815","nonce":4289519146}]},"public_addr":"172.21.15.149:6811/4289519146","cluster_addr":"172.21.15.149:6813/4289519146","heartbeat_back_addr":"172.21.15.149:6817/4289519146","heartbeat_front_addr":"172.21.15.149:6815/4289519146","state":["exists","up"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6819","nonce":2771010477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6821","nonce":2771010477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6825","nonce":2771010477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6823","nonce":2771010477}]},"public_addr":"172.21.15.149:6819/2771010477","cluster_addr":"172.21.15.149:6821/2771010477","heartbeat_back_addr":"172.21.15.149:6825/2771010477","heartbeat_front_addr":"172.21.15.149:6823/2771010477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.313297+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.642861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.654781+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.302606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.385791+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.915266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.121365+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.394432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:15:42.033 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.034+0000 7fe53a5c3640 1 -- 172.21.15.87:0/3305534593 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe520078b90 msgr2=0x7fe52007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:42.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.034+0000 7fe53a5c3640 1 --2- 172.21.15.87:0/3305534593 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe520078b90 0x7fe52007b0a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe524034000 tx=0x7fe524004730 comp rx=0 tx=0).stop 2024-09-17T03:15:42.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.034+0000 7fe53a5c3640 1 -- 172.21.15.87:0/3305534593 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5340d0460 msgr2=0x7fe5341f6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:42.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.034+0000 7fe53a5c3640 1 --2- 172.21.15.87:0/3305534593 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5340d0460 0x7fe5341f6c00 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe528001eb0 tx=0x7fe52800c890 comp rx=0 tx=0).stop 2024-09-17T03:15:42.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.034+0000 7fe53a5c3640 1 -- 172.21.15.87:0/3305534593 shutdown_connections 2024-09-17T03:15:42.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.034+0000 7fe53a5c3640 1 --2- 172.21.15.87:0/3305534593 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fe520078b90 0x7fe52007b0a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:42.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.034+0000 7fe53a5c3640 1 --2- 172.21.15.87:0/3305534593 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fe53415c6c0 0x7fe5340ab050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:42.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.034+0000 7fe53a5c3640 1 --2- 172.21.15.87:0/3305534593 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe5340d0460 0x7fe5341f6c00 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:42.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.034+0000 7fe53a5c3640 1 --2- 172.21.15.87:0/3305534593 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fe5340b3d00 0x7fe534151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:42.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.034+0000 7fe53a5c3640 1 -- 172.21.15.87:0/3305534593 >> 172.21.15.87:0/3305534593 conn(0x7fe534069360 msgr2=0x7fe5340b70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:42.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.034+0000 7fe53a5c3640 1 -- 172.21.15.87:0/3305534593 shutdown_connections 2024-09-17T03:15:42.035 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.034+0000 7fe53a5c3640 1 -- 172.21.15.87:0/3305534593 wait complete. 2024-09-17T03:15:42.045 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T03:15:42.045 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:15:42.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.158+0000 7fb74e6bd640 1 -- 172.21.15.87:0/2043811511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7480ce690 msgr2=0x7fb7480d0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:42.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.158+0000 7fb74e6bd640 1 --2- 172.21.15.87:0/2043811511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7480ce690 0x7fb7480d0b30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb748072400 tx=0x7fb73c02a4d0 comp rx=0 tx=0).stop 2024-09-17T03:15:42.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.159+0000 7fb74e6bd640 1 -- 172.21.15.87:0/2043811511 shutdown_connections 2024-09-17T03:15:42.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.159+0000 7fb74e6bd640 1 --2- 172.21.15.87:0/2043811511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7480ce690 0x7fb7480d0b30 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:42.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.159+0000 7fb74e6bd640 1 --2- 172.21.15.87:0/2043811511 >> v2:172.21.15.149:3300/0 conn(0x7fb748071120 0x7fb74815b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:42.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.159+0000 7fb74e6bd640 1 -- 172.21.15.87:0/2043811511 >> 172.21.15.87:0/2043811511 conn(0x7fb74806cfc0 msgr2=0x7fb74806d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:42.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.159+0000 7fb74e6bd640 1 -- 172.21.15.87:0/2043811511 shutdown_connections 2024-09-17T03:15:42.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.159+0000 7fb74e6bd640 1 -- 172.21.15.87:0/2043811511 wait complete. 2024-09-17T03:15:42.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.160+0000 7fb74e6bd640 1 Processor -- start 2024-09-17T03:15:42.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.161+0000 7fb74e6bd640 1 -- start start 2024-09-17T03:15:42.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.162+0000 7fb74e6bd640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb748071120 0x7fb7481467c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:42.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.162+0000 7fb74e6bd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7480ce690 0x7fb74815a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:42.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.162+0000 7fb74e6bd640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7480d0250 0x7fb7481548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:42.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.162+0000 7fb74e6bd640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb7480d1480 con 0x7fb7480ce690 2024-09-17T03:15:42.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.162+0000 7fb74e6bd640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb7480d1300 con 0x7fb7480d0250 2024-09-17T03:15:42.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.162+0000 7fb74e6bd640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb74815f7d0 con 0x7fb748071120 2024-09-17T03:15:42.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.162+0000 7fb7477fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7480ce690 0x7fb74815a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:42.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.162+0000 7fb7477fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7480ce690 0x7fb74815a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36100/0 (socket says 172.21.15.87:36100) 2024-09-17T03:15:42.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.162+0000 7fb7477fe640 1 -- 172.21.15.87:0/3101126917 learned_addr learned my addr 172.21.15.87:0/3101126917 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:42.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.162+0000 7fb747fff640 1 --2- 172.21.15.87:0/3101126917 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb748071120 0x7fb7481467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:42.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.162+0000 7fb74cc5a640 1 --2- 172.21.15.87:0/3101126917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7480d0250 0x7fb7481548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:42.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.162+0000 7fb7477fe640 1 -- 172.21.15.87:0/3101126917 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb748071120 msgr2=0x7fb7481467c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:42.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.162+0000 7fb7477fe640 1 --2- 172.21.15.87:0/3101126917 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb748071120 0x7fb7481467c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:42.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.162+0000 7fb7477fe640 1 -- 172.21.15.87:0/3101126917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7480d0250 msgr2=0x7fb7481548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:42.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.162+0000 7fb7477fe640 1 --2- 172.21.15.87:0/3101126917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7480d0250 0x7fb7481548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:42.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.162+0000 7fb7477fe640 1 -- 172.21.15.87:0/3101126917 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb748153160 con 0x7fb7480ce690 2024-09-17T03:15:42.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.162+0000 7fb747fff640 1 --2- 172.21.15.87:0/3101126917 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb748071120 0x7fb7481467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:42.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.162+0000 7fb7477fe640 1 --2- 172.21.15.87:0/3101126917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7480ce690 0x7fb74815a380 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb73c034040 tx=0x7fb73c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:42.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.163+0000 7fb744ff9640 1 -- 172.21.15.87:0/3101126917 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73c03e030 con 0x7fb7480ce690 2024-09-17T03:15:42.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.163+0000 7fb744ff9640 1 -- 172.21.15.87:0/3101126917 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb73c003d10 con 0x7fb7480ce690 2024-09-17T03:15:42.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.163+0000 7fb74e6bd640 1 -- 172.21.15.87:0/3101126917 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb748153440 con 0x7fb7480ce690 2024-09-17T03:15:42.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.163+0000 7fb744ff9640 1 -- 172.21.15.87:0/3101126917 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73c035040 con 0x7fb7480ce690 2024-09-17T03:15:42.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.163+0000 7fb74e6bd640 1 -- 172.21.15.87:0/3101126917 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb748153a00 con 0x7fb7480ce690 2024-09-17T03:15:42.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.164+0000 7fb74e6bd640 1 -- 172.21.15.87:0/3101126917 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb74815fc90 con 0x7fb7480ce690 2024-09-17T03:15:42.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.164+0000 7fb744ff9640 1 -- 172.21.15.87:0/3101126917 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb73c041030 con 0x7fb7480ce690 2024-09-17T03:15:42.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.165+0000 7fb744ff9640 1 --2- 172.21.15.87:0/3101126917 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb734078b90 0x7fb73407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:42.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.165+0000 7fb744ff9640 1 -- 172.21.15.87:0/3101126917 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fb73c0bb4c0 con 0x7fb7480ce690 2024-09-17T03:15:42.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.165+0000 7fb747fff640 1 --2- 172.21.15.87:0/3101126917 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb734078b90 0x7fb73407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:42.165 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.166+0000 7fb747fff640 1 --2- 172.21.15.87:0/3101126917 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb734078b90 0x7fb73407b0a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb738009c50 tx=0x7fb738005f30 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:42.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.169+0000 7fb744ff9640 1 -- 172.21.15.87:0/3101126917 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb73c083900 con 0x7fb7480ce690 2024-09-17T03:15:42.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.340+0000 7fb74e6bd640 1 -- 172.21.15.87:0/3101126917 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb74815ae10 con 0x7fb734078b90 2024-09-17T03:15:42.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.345+0000 7fb744ff9640 1 -- 172.21.15.87:0/3101126917 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241868 (secure 0 0 0) 0x7fb74815ae10 con 0x7fb734078b90 2024-09-17T03:15:42.346 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:42.347 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-09-17T03:15:42.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.348+0000 7fb74e6bd640 1 -- 172.21.15.87:0/3101126917 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb734078b90 msgr2=0x7fb73407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:42.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.348+0000 7fb74e6bd640 1 --2- 172.21.15.87:0/3101126917 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb734078b90 0x7fb73407b0a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb738009c50 tx=0x7fb738005f30 comp rx=0 tx=0).stop 2024-09-17T03:15:42.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.348+0000 7fb74e6bd640 1 -- 172.21.15.87:0/3101126917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7480ce690 msgr2=0x7fb74815a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:42.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.348+0000 7fb74e6bd640 1 --2- 172.21.15.87:0/3101126917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7480ce690 0x7fb74815a380 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb73c034040 tx=0x7fb73c003820 comp rx=0 tx=0).stop 2024-09-17T03:15:42.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.348+0000 7fb74e6bd640 1 -- 172.21.15.87:0/3101126917 shutdown_connections 2024-09-17T03:15:42.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.348+0000 7fb74e6bd640 1 --2- 172.21.15.87:0/3101126917 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb734078b90 0x7fb73407b0a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:42.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.348+0000 7fb74e6bd640 1 --2- 172.21.15.87:0/3101126917 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb7480d0250 0x7fb7481548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:42.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.348+0000 7fb74e6bd640 1 --2- 172.21.15.87:0/3101126917 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7480ce690 0x7fb74815a380 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:42.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.348+0000 7fb74e6bd640 1 --2- 172.21.15.87:0/3101126917 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb748071120 0x7fb7481467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:42.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.348+0000 7fb74e6bd640 1 -- 172.21.15.87:0/3101126917 >> 172.21.15.87:0/3101126917 conn(0x7fb74806cfc0 msgr2=0x7fb7480b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:42.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.348+0000 7fb74e6bd640 1 -- 172.21.15.87:0/3101126917 shutdown_connections 2024-09-17T03:15:42.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.348+0000 7fb74e6bd640 1 -- 172.21.15.87:0/3101126917 wait complete. 2024-09-17T03:15:42.358 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2024-09-17T03:15:41.438402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":86,"acting":86,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":35,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229972,"kb_used_data":5388,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744748,"statfs":{"total":773094113280,"available":772858621952,"internally_reserved":0,"allocated":5517312,"data_stored":2922349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.099408"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.377499+0000","last_change":"2024-09-17T03:15:39.368117+0000","last_active":"2024-09-17T03:15:39.377499+0000","last_peered":"2024-09-17T03:15:39.377499+0000","last_clean":"2024-09-17T03:15:39.377499+0000","last_became_active":"2024-09-17T03:15:39.366527+0000","last_became_peered":"2024-09-17T03:15:39.366527+0000","last_unstale":"2024-09-17T03:15:39.377499+0000","last_undegraded":"2024-09-17T03:15:39.377499+0000","last_fullsized":"2024-09-17T03:15:39.377499+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:00:48.160980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.377520+0000","last_change":"2024-09-17T03:15:39.367252+0000","last_active":"2024-09-17T03:15:39.377520+0000","last_peered":"2024-09-17T03:15:39.377520+0000","last_clean":"2024-09-17T03:15:39.377520+0000","last_became_active":"2024-09-17T03:15:39.366982+0000","last_became_peered":"2024-09-17T03:15:39.366982+0000","last_unstale":"2024-09-17T03:15:39.377520+0000","last_undegraded":"2024-09-17T03:15:39.377520+0000","last_fullsized":"2024-09-17T03:15:39.377520+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:02:23.197700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.369870+0000","last_change":"2024-09-17T03:15:39.366135+0000","last_active":"2024-09-17T03:15:39.369870+0000","last_peered":"2024-09-17T03:15:39.369870+0000","last_clean":"2024-09-17T03:15:39.369870+0000","last_became_active":"2024-09-17T03:15:39.365800+0000","last_became_peered":"2024-09-17T03:15:39.365800+0000","last_unstale":"2024-09-17T03:15:39.369870+0000","last_undegraded":"2024-09-17T03:15:39.369870+0000","last_fullsized":"2024-09-17T03:15:39.369870+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:17:16.473419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.380167+0000","last_change":"2024-09-17T03:15:39.371434+0000","last_active":"2024-09-17T03:15:39.380167+0000","last_peered":"2024-09-17T03:15:39.380167+0000","last_clean":"2024-09-17T03:15:39.380167+0000","last_became_active":"2024-09-17T03:15:39.371098+0000","last_became_peered":"2024-09-17T03:15:39.371098+0000","last_unstale":"2024-09-17T03:15:39.380167+0000","last_undegraded":"2024-09-17T03:15:39.380167+0000","last_fullsized":"2024-09-17T03:15:39.380167+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:33.488929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.380327+0000","last_change":"2024-09-17T03:15:39.375041+0000","last_active":"2024-09-17T03:15:39.380327+0000","last_peered":"2024-09-17T03:15:39.380327+0000","last_clean":"2024-09-17T03:15:39.380327+0000","last_became_active":"2024-09-17T03:15:39.374237+0000","last_became_peered":"2024-09-17T03:15:39.374237+0000","last_unstale":"2024-09-17T03:15:39.380327+0000","last_undegraded":"2024-09-17T03:15:39.380327+0000","last_fullsized":"2024-09-17T03:15:39.380327+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:06.672069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.377585+0000","last_change":"2024-09-17T03:15:39.368629+0000","last_active":"2024-09-17T03:15:39.377585+0000","last_peered":"2024-09-17T03:15:39.377585+0000","last_clean":"2024-09-17T03:15:39.377585+0000","last_became_active":"2024-09-17T03:15:39.368078+0000","last_became_peered":"2024-09-17T03:15:39.368078+0000","last_unstale":"2024-09-17T03:15:39.377585+0000","last_undegraded":"2024-09-17T03:15:39.377585+0000","last_fullsized":"2024-09-17T03:15:39.377585+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:24.126150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.380304+0000","last_change":"2024-09-17T03:15:39.374436+0000","last_active":"2024-09-17T03:15:39.380304+0000","last_peered":"2024-09-17T03:15:39.380304+0000","last_clean":"2024-09-17T03:15:39.380304+0000","last_became_active":"2024-09-17T03:15:39.372074+0000","last_became_peered":"2024-09-17T03:15:39.372074+0000","last_unstale":"2024-09-17T03:15:39.380304+0000","last_undegraded":"2024-09-17T03:15:39.380304+0000","last_fullsized":"2024-09-17T03:15:39.380304+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:48:37.306791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.377548+0000","last_change":"2024-09-17T03:15:39.367577+0000","last_active":"2024-09-17T03:15:39.377548+0000","last_peered":"2024-09-17T03:15:39.377548+0000","last_clean":"2024-09-17T03:15:39.377548+0000","last_became_active":"2024-09-17T03:15:39.367310+0000","last_became_peered":"2024-09-17T03:15:39.367310+0000","last_unstale":"2024-09-17T03:15:39.377548+0000","last_undegraded":"2024-09-17T03:15:39.377548+0000","last_fullsized":"2024-09-17T03:15:39.377548+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:57:05.939726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.347030+0000","last_change":"2024-09-17T03:15:37.347030+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.347030+0000","last_undegraded":"2024-09-17T03:15:37.347030+0000","last_fullsized":"2024-09-17T03:15:37.347030+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.344765+0000","last_change":"2024-09-17T03:15:37.344765+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.344765+0000","last_undegraded":"2024-09-17T03:15:37.344765+0000","last_fullsized":"2024-09-17T03:15:37.344765+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.342115+0000","last_change":"2024-09-17T03:15:37.342115+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.342115+0000","last_undegraded":"2024-09-17T03:15:37.342115+0000","last_fullsized":"2024-09-17T03:15:37.342115+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.343314+0000","last_change":"2024-09-17T03:15:37.343314+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.343314+0000","last_undegraded":"2024-09-17T03:15:37.343314+0000","last_fullsized":"2024-09-17T03:15:37.343314+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"49'1","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.384912+0000","last_change":"2024-09-17T03:15:38.358694+0000","last_active":"2024-09-17T03:15:39.384912+0000","last_peered":"2024-09-17T03:15:39.384912+0000","last_clean":"2024-09-17T03:15:39.384912+0000","last_became_active":"2024-09-17T03:15:38.358307+0000","last_became_peered":"2024-09-17T03:15:38.358307+0000","last_unstale":"2024-09-17T03:15:39.384912+0000","last_undegraded":"2024-09-17T03:15:39.384912+0000","last_fullsized":"2024-09-17T03:15:39.384912+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:15.429280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.347253+0000","last_change":"2024-09-17T03:15:37.347253+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.347253+0000","last_undegraded":"2024-09-17T03:15:37.347253+0000","last_fullsized":"2024-09-17T03:15:37.347253+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.347297+0000","last_change":"2024-09-17T03:15:37.347297+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.347297+0000","last_undegraded":"2024-09-17T03:15:37.347297+0000","last_fullsized":"2024-09-17T03:15:37.347297+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.353431+0000","last_change":"2024-09-17T03:15:38.362750+0000","last_active":"2024-09-17T03:15:39.353431+0000","last_peered":"2024-09-17T03:15:39.353431+0000","last_clean":"2024-09-17T03:15:39.353431+0000","last_became_active":"2024-09-17T03:15:38.362412+0000","last_became_peered":"2024-09-17T03:15:38.362412+0000","last_unstale":"2024-09-17T03:15:39.353431+0000","last_undegraded":"2024-09-17T03:15:39.353431+0000","last_fullsized":"2024-09-17T03:15:39.353431+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:40:22.976291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.348269+0000","last_change":"2024-09-17T03:15:37.348269+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.348269+0000","last_undegraded":"2024-09-17T03:15:37.348269+0000","last_fullsized":"2024-09-17T03:15:37.348269+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.348206+0000","last_change":"2024-09-17T03:15:37.348206+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.348206+0000","last_undegraded":"2024-09-17T03:15:37.348206+0000","last_fullsized":"2024-09-17T03:15:37.348206+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.347947+0000","last_change":"2024-09-17T03:15:37.347947+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.347947+0000","last_undegraded":"2024-09-17T03:15:37.347947+0000","last_fullsized":"2024-09-17T03:15:37.347947+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.342227+0000","last_change":"2024-09-17T03:15:37.342227+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.342227+0000","last_undegraded":"2024-09-17T03:15:37.342227+0000","last_fullsized":"2024-09-17T03:15:37.342227+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.340392+0000","last_change":"2024-09-17T03:15:37.340392+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.340392+0000","last_undegraded":"2024-09-17T03:15:37.340392+0000","last_fullsized":"2024-09-17T03:15:37.340392+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.347002+0000","last_change":"2024-09-17T03:15:37.347002+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.347002+0000","last_undegraded":"2024-09-17T03:15:37.347002+0000","last_fullsized":"2024-09-17T03:15:37.347002+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.343266+0000","last_change":"2024-09-17T03:15:37.343266+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.343266+0000","last_undegraded":"2024-09-17T03:15:37.343266+0000","last_fullsized":"2024-09-17T03:15:37.343266+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.380294+0000","last_change":"2024-09-17T03:15:39.374972+0000","last_active":"2024-09-17T03:15:39.380294+0000","last_peered":"2024-09-17T03:15:39.380294+0000","last_clean":"2024-09-17T03:15:39.380294+0000","last_became_active":"2024-09-17T03:15:39.374418+0000","last_became_peered":"2024-09-17T03:15:39.374418+0000","last_unstale":"2024-09-17T03:15:39.380294+0000","last_undegraded":"2024-09-17T03:15:39.380294+0000","last_fullsized":"2024-09-17T03:15:39.380294+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:53:49.563161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.340186+0000","last_change":"2024-09-17T03:15:37.340186+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.340186+0000","last_undegraded":"2024-09-17T03:15:37.340186+0000","last_fullsized":"2024-09-17T03:15:37.340186+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.340193+0000","last_change":"2024-09-17T03:15:37.340193+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.340193+0000","last_undegraded":"2024-09-17T03:15:37.340193+0000","last_fullsized":"2024-09-17T03:15:37.340193+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.340448+0000","last_change":"2024-09-17T03:15:37.340448+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.340448+0000","last_undegraded":"2024-09-17T03:15:37.340448+0000","last_fullsized":"2024-09-17T03:15:37.340448+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.348521+0000","last_change":"2024-09-17T03:15:37.348521+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.348521+0000","last_undegraded":"2024-09-17T03:15:37.348521+0000","last_fullsized":"2024-09-17T03:15:37.348521+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.342484+0000","last_change":"2024-09-17T03:15:37.342484+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.342484+0000","last_undegraded":"2024-09-17T03:15:37.342484+0000","last_fullsized":"2024-09-17T03:15:37.342484+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.345700+0000","last_change":"2024-09-17T03:15:37.345700+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.345700+0000","last_undegraded":"2024-09-17T03:15:37.345700+0000","last_fullsized":"2024-09-17T03:15:37.345700+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.357849+0000","last_change":"2024-09-17T03:15:38.361994+0000","last_active":"2024-09-17T03:15:39.357849+0000","last_peered":"2024-09-17T03:15:39.357849+0000","last_clean":"2024-09-17T03:15:39.357849+0000","last_became_active":"2024-09-17T03:15:38.361661+0000","last_became_peered":"2024-09-17T03:15:38.361661+0000","last_unstale":"2024-09-17T03:15:39.357849+0000","last_undegraded":"2024-09-17T03:15:39.357849+0000","last_fullsized":"2024-09-17T03:15:39.357849+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:46:31.452976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.341965+0000","last_change":"2024-09-17T03:15:37.341965+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.341965+0000","last_undegraded":"2024-09-17T03:15:37.341965+0000","last_fullsized":"2024-09-17T03:15:37.341965+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.342669+0000","last_change":"2024-09-17T03:15:37.342669+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.342669+0000","last_undegraded":"2024-09-17T03:15:37.342669+0000","last_fullsized":"2024-09-17T03:15:37.342669+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.347427+0000","last_change":"2024-09-17T03:15:37.347427+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.347427+0000","last_undegraded":"2024-09-17T03:15:37.347427+0000","last_fullsized":"2024-09-17T03:15:37.347427+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.347233+0000","last_change":"2024-09-17T03:15:37.347233+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.347233+0000","last_undegraded":"2024-09-17T03:15:37.347233+0000","last_fullsized":"2024-09-17T03:15:37.347233+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.375247+0000","last_change":"2024-09-17T03:15:39.366303+0000","last_active":"2024-09-17T03:15:39.375247+0000","last_peered":"2024-09-17T03:15:39.375247+0000","last_clean":"2024-09-17T03:15:39.375247+0000","last_became_active":"2024-09-17T03:15:39.365953+0000","last_became_peered":"2024-09-17T03:15:39.365953+0000","last_unstale":"2024-09-17T03:15:39.375247+0000","last_undegraded":"2024-09-17T03:15:39.375247+0000","last_fullsized":"2024-09-17T03:15:39.375247+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:51.052810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.356414+0000","last_change":"2024-09-17T03:15:38.362902+0000","last_active":"2024-09-17T03:15:39.356414+0000","last_peered":"2024-09-17T03:15:39.356414+0000","last_clean":"2024-09-17T03:15:39.356414+0000","last_became_active":"2024-09-17T03:15:38.362598+0000","last_became_peered":"2024-09-17T03:15:38.362598+0000","last_unstale":"2024-09-17T03:15:39.356414+0000","last_undegraded":"2024-09-17T03:15:39.356414+0000","last_fullsized":"2024-09-17T03:15:39.356414+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:56.538985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.341949+0000","last_change":"2024-09-17T03:15:37.341949+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.341949+0000","last_undegraded":"2024-09-17T03:15:37.341949+0000","last_fullsized":"2024-09-17T03:15:37.341949+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.358479+0000","last_change":"2024-09-17T03:15:38.361181+0000","last_active":"2024-09-17T03:15:39.358479+0000","last_peered":"2024-09-17T03:15:39.358479+0000","last_clean":"2024-09-17T03:15:39.358479+0000","last_became_active":"2024-09-17T03:15:38.360468+0000","last_became_peered":"2024-09-17T03:15:38.360468+0000","last_unstale":"2024-09-17T03:15:39.358479+0000","last_undegraded":"2024-09-17T03:15:39.358479+0000","last_fullsized":"2024-09-17T03:15:39.358479+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:52.635369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.342475+0000","last_change":"2024-09-17T03:15:37.342475+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.342475+0000","last_undegraded":"2024-09-17T03:15:37.342475+0000","last_fullsized":"2024-09-17T03:15:37.342475+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.352546+0000","last_change":"2024-09-17T03:15:37.352546+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.352546+0000","last_undegraded":"2024-09-17T03:15:37.352546+0000","last_fullsized":"2024-09-17T03:15:37.352546+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.343143+0000","last_change":"2024-09-17T03:15:37.343143+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.343143+0000","last_undegraded":"2024-09-17T03:15:37.343143+0000","last_fullsized":"2024-09-17T03:15:37.343143+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":153,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T03:15:37.353340+0000","last_change":"2024-09-17T03:14:04.862167+0000","last_active":"2024-09-17T03:15:37.353340+0000","last_peered":"2024-09-17T03:15:37.353340+0000","last_clean":"2024-09-17T03:15:37.353340+0000","last_became_active":"2024-09-17T03:14:04.861731+0000","last_became_peered":"2024-09-17T03:14:04.861731+0000","last_unstale":"2024-09-17T03:15:37.353340+0000","last_undegraded":"2024-09-17T03:15:37.353340+0000","last_fullsized":"2024-09-17T03:15:37.353340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_clean_scrub_stamp":"2024-09-17T03:14:03.847721+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:32.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":1},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":536576,"data_stored":215463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":536576,"data_stored":215463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509463,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1142784,"data_stored":811648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":536576,"data_stored":215463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":536576,"data_stored":215463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509464,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28140,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343700,"statfs":{"total":96636764160,"available":96607948800,"internally_reserved":0,"allocated":1130496,"data_stored":805831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":536576,"data_stored":215463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":561152,"data_stored":227555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:15:42.363 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:15:42.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.511+0000 7f8617b9b640 1 -- 172.21.15.87:0/1573641884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86100b76d0 msgr2=0x7f86100cee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:42.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.511+0000 7f8617b9b640 1 --2- 172.21.15.87:0/1573641884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86100b76d0 0x7f86100cee00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8600009830 tx=0x7f860002a510 comp rx=0 tx=0).stop 2024-09-17T03:15:42.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.511+0000 7f8617b9b640 1 -- 172.21.15.87:0/1573641884 shutdown_connections 2024-09-17T03:15:42.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.511+0000 7f8617b9b640 1 --2- 172.21.15.87:0/1573641884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86100b76d0 0x7f86100cee00 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:42.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.511+0000 7f8617b9b640 1 -- 172.21.15.87:0/1573641884 >> 172.21.15.87:0/1573641884 conn(0x7f86100d2230 msgr2=0x7f86100d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:42.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.511+0000 7f8617b9b640 1 -- 172.21.15.87:0/1573641884 shutdown_connections 2024-09-17T03:15:42.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.511+0000 7f8617b9b640 1 -- 172.21.15.87:0/1573641884 wait complete. 2024-09-17T03:15:42.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.513+0000 7f8617b9b640 1 Processor -- start 2024-09-17T03:15:42.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.513+0000 7f8617b9b640 1 -- start start 2024-09-17T03:15:42.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.514+0000 7f8617b9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86100b76d0 0x7f8610158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:42.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.514+0000 7f8617b9b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86100ccf80 0x7f861015c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:42.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.514+0000 7f8617b9b640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f86100cf340 0x7f861015c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:42.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.514+0000 7f8617b9b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f86100d13d0 con 0x7f86100b76d0 2024-09-17T03:15:42.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.514+0000 7f8617b9b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f86100d1250 con 0x7f86100ccf80 2024-09-17T03:15:42.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.514+0000 7f8617b9b640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f8610168120 con 0x7f86100cf340 2024-09-17T03:15:42.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.514+0000 7f8615937640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86100b76d0 0x7f8610158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:42.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.515+0000 7f8615937640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86100b76d0 0x7f8610158020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36122/0 (socket says 172.21.15.87:36122) 2024-09-17T03:15:42.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.515+0000 7f8615937640 1 -- 172.21.15.87:0/479674591 learned_addr learned my addr 172.21.15.87:0/479674591 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:42.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.515+0000 7f8615136640 1 --2- 172.21.15.87:0/479674591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86100ccf80 0x7f861015c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:42.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.515+0000 7f8616138640 1 --2- 172.21.15.87:0/479674591 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f86100cf340 0x7f861015c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:42.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.515+0000 7f8615937640 1 -- 172.21.15.87:0/479674591 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f86100cf340 msgr2=0x7f861015c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:42.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.515+0000 7f8615937640 1 --2- 172.21.15.87:0/479674591 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f86100cf340 0x7f861015c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:42.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.515+0000 7f8615937640 1 -- 172.21.15.87:0/479674591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86100ccf80 msgr2=0x7f861015c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:42.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.515+0000 7f8615937640 1 --2- 172.21.15.87:0/479674591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86100ccf80 0x7f861015c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:42.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.515+0000 7f8615937640 1 -- 172.21.15.87:0/479674591 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86101f6e70 con 0x7f86100b76d0 2024-09-17T03:15:42.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.515+0000 7f8616138640 1 --2- 172.21.15.87:0/479674591 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f86100cf340 0x7f861015c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:42.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.515+0000 7f8615136640 1 --2- 172.21.15.87:0/479674591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86100ccf80 0x7f861015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:42.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.515+0000 7f8615937640 1 --2- 172.21.15.87:0/479674591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86100b76d0 0x7f8610158020 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f860002ab00 tx=0x7f8600033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:42.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.515+0000 7f860e7fc640 1 -- 172.21.15.87:0/479674591 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600005c90 con 0x7f86100b76d0 2024-09-17T03:15:42.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.515+0000 7f860e7fc640 1 -- 172.21.15.87:0/479674591 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8600031e50 con 0x7f86100b76d0 2024-09-17T03:15:42.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.515+0000 7f8617b9b640 1 -- 172.21.15.87:0/479674591 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86101f7150 con 0x7f86100b76d0 2024-09-17T03:15:42.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.515+0000 7f860e7fc640 1 -- 172.21.15.87:0/479674591 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600005260 con 0x7f86100b76d0 2024-09-17T03:15:42.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.515+0000 7f8617b9b640 1 -- 172.21.15.87:0/479674591 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86101f7710 con 0x7f86100b76d0 2024-09-17T03:15:42.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.516+0000 7f8617b9b640 1 -- 172.21.15.87:0/479674591 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86100cee00 con 0x7f86100b76d0 2024-09-17T03:15:42.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.517+0000 7f860e7fc640 1 -- 172.21.15.87:0/479674591 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8600005400 con 0x7f86100b76d0 2024-09-17T03:15:42.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.517+0000 7f860e7fc640 1 --2- 172.21.15.87:0/479674591 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f85fc078b90 0x7f85fc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:42.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.517+0000 7f860e7fc640 1 -- 172.21.15.87:0/479674591 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f86000bbec0 con 0x7f86100b76d0 2024-09-17T03:15:42.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.518+0000 7f8615136640 1 --2- 172.21.15.87:0/479674591 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f85fc078b90 0x7f85fc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:42.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.518+0000 7f8615136640 1 --2- 172.21.15.87:0/479674591 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f85fc078b90 0x7f85fc07b0a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8610164640 tx=0x7f8604009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:42.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.522+0000 7f860e7fc640 1 -- 172.21.15.87:0/479674591 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f860003f030 con 0x7f86100b76d0 2024-09-17T03:15:42.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.693+0000 7f8617b9b640 1 -- 172.21.15.87:0/479674591 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f86100b74b0 con 0x7f85fc078b90 2024-09-17T03:15:42.696 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.697+0000 7f860e7fc640 1 -- 172.21.15.87:0/479674591 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241868 (secure 0 0 0) 0x7f86100b74b0 con 0x7f85fc078b90 2024-09-17T03:15:42.698 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:42.699 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-09-17T03:15:42.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.700+0000 7f8617b9b640 1 -- 172.21.15.87:0/479674591 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f85fc078b90 msgr2=0x7f85fc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:42.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.700+0000 7f8617b9b640 1 --2- 172.21.15.87:0/479674591 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f85fc078b90 0x7f85fc07b0a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8610164640 tx=0x7f8604009040 comp rx=0 tx=0).stop 2024-09-17T03:15:42.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.700+0000 7f8617b9b640 1 -- 172.21.15.87:0/479674591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86100b76d0 msgr2=0x7f8610158020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:42.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.700+0000 7f8617b9b640 1 --2- 172.21.15.87:0/479674591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86100b76d0 0x7f8610158020 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f860002ab00 tx=0x7f8600033c20 comp rx=0 tx=0).stop 2024-09-17T03:15:42.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.700+0000 7f8617b9b640 1 -- 172.21.15.87:0/479674591 shutdown_connections 2024-09-17T03:15:42.701 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.700+0000 7f8617b9b640 1 --2- 172.21.15.87:0/479674591 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f85fc078b90 0x7f85fc07b0a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:42.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.700+0000 7f8617b9b640 1 --2- 172.21.15.87:0/479674591 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f86100cf340 0x7f861015c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:42.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.700+0000 7f8617b9b640 1 --2- 172.21.15.87:0/479674591 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f86100ccf80 0x7f861015c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:42.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.700+0000 7f8617b9b640 1 --2- 172.21.15.87:0/479674591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86100b76d0 0x7f8610158020 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:42.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.700+0000 7f8617b9b640 1 -- 172.21.15.87:0/479674591 >> 172.21.15.87:0/479674591 conn(0x7f86100d2230 msgr2=0x7f86100d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:42.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.700+0000 7f8617b9b640 1 -- 172.21.15.87:0/479674591 shutdown_connections 2024-09-17T03:15:42.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.700+0000 7f8617b9b640 1 -- 172.21.15.87:0/479674591 wait complete. 2024-09-17T03:15:42.710 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2024-09-17T03:15:41.438402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":86,"acting":86,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":35,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229972,"kb_used_data":5388,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744748,"statfs":{"total":773094113280,"available":772858621952,"internally_reserved":0,"allocated":5517312,"data_stored":2922349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.099408"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.377499+0000","last_change":"2024-09-17T03:15:39.368117+0000","last_active":"2024-09-17T03:15:39.377499+0000","last_peered":"2024-09-17T03:15:39.377499+0000","last_clean":"2024-09-17T03:15:39.377499+0000","last_became_active":"2024-09-17T03:15:39.366527+0000","last_became_peered":"2024-09-17T03:15:39.366527+0000","last_unstale":"2024-09-17T03:15:39.377499+0000","last_undegraded":"2024-09-17T03:15:39.377499+0000","last_fullsized":"2024-09-17T03:15:39.377499+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:00:48.160980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.377520+0000","last_change":"2024-09-17T03:15:39.367252+0000","last_active":"2024-09-17T03:15:39.377520+0000","last_peered":"2024-09-17T03:15:39.377520+0000","last_clean":"2024-09-17T03:15:39.377520+0000","last_became_active":"2024-09-17T03:15:39.366982+0000","last_became_peered":"2024-09-17T03:15:39.366982+0000","last_unstale":"2024-09-17T03:15:39.377520+0000","last_undegraded":"2024-09-17T03:15:39.377520+0000","last_fullsized":"2024-09-17T03:15:39.377520+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:02:23.197700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.369870+0000","last_change":"2024-09-17T03:15:39.366135+0000","last_active":"2024-09-17T03:15:39.369870+0000","last_peered":"2024-09-17T03:15:39.369870+0000","last_clean":"2024-09-17T03:15:39.369870+0000","last_became_active":"2024-09-17T03:15:39.365800+0000","last_became_peered":"2024-09-17T03:15:39.365800+0000","last_unstale":"2024-09-17T03:15:39.369870+0000","last_undegraded":"2024-09-17T03:15:39.369870+0000","last_fullsized":"2024-09-17T03:15:39.369870+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:17:16.473419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.380167+0000","last_change":"2024-09-17T03:15:39.371434+0000","last_active":"2024-09-17T03:15:39.380167+0000","last_peered":"2024-09-17T03:15:39.380167+0000","last_clean":"2024-09-17T03:15:39.380167+0000","last_became_active":"2024-09-17T03:15:39.371098+0000","last_became_peered":"2024-09-17T03:15:39.371098+0000","last_unstale":"2024-09-17T03:15:39.380167+0000","last_undegraded":"2024-09-17T03:15:39.380167+0000","last_fullsized":"2024-09-17T03:15:39.380167+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:33.488929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.380327+0000","last_change":"2024-09-17T03:15:39.375041+0000","last_active":"2024-09-17T03:15:39.380327+0000","last_peered":"2024-09-17T03:15:39.380327+0000","last_clean":"2024-09-17T03:15:39.380327+0000","last_became_active":"2024-09-17T03:15:39.374237+0000","last_became_peered":"2024-09-17T03:15:39.374237+0000","last_unstale":"2024-09-17T03:15:39.380327+0000","last_undegraded":"2024-09-17T03:15:39.380327+0000","last_fullsized":"2024-09-17T03:15:39.380327+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:06.672069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.377585+0000","last_change":"2024-09-17T03:15:39.368629+0000","last_active":"2024-09-17T03:15:39.377585+0000","last_peered":"2024-09-17T03:15:39.377585+0000","last_clean":"2024-09-17T03:15:39.377585+0000","last_became_active":"2024-09-17T03:15:39.368078+0000","last_became_peered":"2024-09-17T03:15:39.368078+0000","last_unstale":"2024-09-17T03:15:39.377585+0000","last_undegraded":"2024-09-17T03:15:39.377585+0000","last_fullsized":"2024-09-17T03:15:39.377585+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:24.126150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.380304+0000","last_change":"2024-09-17T03:15:39.374436+0000","last_active":"2024-09-17T03:15:39.380304+0000","last_peered":"2024-09-17T03:15:39.380304+0000","last_clean":"2024-09-17T03:15:39.380304+0000","last_became_active":"2024-09-17T03:15:39.372074+0000","last_became_peered":"2024-09-17T03:15:39.372074+0000","last_unstale":"2024-09-17T03:15:39.380304+0000","last_undegraded":"2024-09-17T03:15:39.380304+0000","last_fullsized":"2024-09-17T03:15:39.380304+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:48:37.306791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.377548+0000","last_change":"2024-09-17T03:15:39.367577+0000","last_active":"2024-09-17T03:15:39.377548+0000","last_peered":"2024-09-17T03:15:39.377548+0000","last_clean":"2024-09-17T03:15:39.377548+0000","last_became_active":"2024-09-17T03:15:39.367310+0000","last_became_peered":"2024-09-17T03:15:39.367310+0000","last_unstale":"2024-09-17T03:15:39.377548+0000","last_undegraded":"2024-09-17T03:15:39.377548+0000","last_fullsized":"2024-09-17T03:15:39.377548+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:57:05.939726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.347030+0000","last_change":"2024-09-17T03:15:37.347030+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.347030+0000","last_undegraded":"2024-09-17T03:15:37.347030+0000","last_fullsized":"2024-09-17T03:15:37.347030+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.344765+0000","last_change":"2024-09-17T03:15:37.344765+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.344765+0000","last_undegraded":"2024-09-17T03:15:37.344765+0000","last_fullsized":"2024-09-17T03:15:37.344765+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.342115+0000","last_change":"2024-09-17T03:15:37.342115+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.342115+0000","last_undegraded":"2024-09-17T03:15:37.342115+0000","last_fullsized":"2024-09-17T03:15:37.342115+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.343314+0000","last_change":"2024-09-17T03:15:37.343314+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.343314+0000","last_undegraded":"2024-09-17T03:15:37.343314+0000","last_fullsized":"2024-09-17T03:15:37.343314+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"49'1","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.384912+0000","last_change":"2024-09-17T03:15:38.358694+0000","last_active":"2024-09-17T03:15:39.384912+0000","last_peered":"2024-09-17T03:15:39.384912+0000","last_clean":"2024-09-17T03:15:39.384912+0000","last_became_active":"2024-09-17T03:15:38.358307+0000","last_became_peered":"2024-09-17T03:15:38.358307+0000","last_unstale":"2024-09-17T03:15:39.384912+0000","last_undegraded":"2024-09-17T03:15:39.384912+0000","last_fullsized":"2024-09-17T03:15:39.384912+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:15.429280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.347253+0000","last_change":"2024-09-17T03:15:37.347253+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.347253+0000","last_undegraded":"2024-09-17T03:15:37.347253+0000","last_fullsized":"2024-09-17T03:15:37.347253+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.347297+0000","last_change":"2024-09-17T03:15:37.347297+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.347297+0000","last_undegraded":"2024-09-17T03:15:37.347297+0000","last_fullsized":"2024-09-17T03:15:37.347297+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.353431+0000","last_change":"2024-09-17T03:15:38.362750+0000","last_active":"2024-09-17T03:15:39.353431+0000","last_peered":"2024-09-17T03:15:39.353431+0000","last_clean":"2024-09-17T03:15:39.353431+0000","last_became_active":"2024-09-17T03:15:38.362412+0000","last_became_peered":"2024-09-17T03:15:38.362412+0000","last_unstale":"2024-09-17T03:15:39.353431+0000","last_undegraded":"2024-09-17T03:15:39.353431+0000","last_fullsized":"2024-09-17T03:15:39.353431+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:40:22.976291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.348269+0000","last_change":"2024-09-17T03:15:37.348269+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.348269+0000","last_undegraded":"2024-09-17T03:15:37.348269+0000","last_fullsized":"2024-09-17T03:15:37.348269+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.348206+0000","last_change":"2024-09-17T03:15:37.348206+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.348206+0000","last_undegraded":"2024-09-17T03:15:37.348206+0000","last_fullsized":"2024-09-17T03:15:37.348206+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.347947+0000","last_change":"2024-09-17T03:15:37.347947+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.347947+0000","last_undegraded":"2024-09-17T03:15:37.347947+0000","last_fullsized":"2024-09-17T03:15:37.347947+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.342227+0000","last_change":"2024-09-17T03:15:37.342227+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.342227+0000","last_undegraded":"2024-09-17T03:15:37.342227+0000","last_fullsized":"2024-09-17T03:15:37.342227+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.340392+0000","last_change":"2024-09-17T03:15:37.340392+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.340392+0000","last_undegraded":"2024-09-17T03:15:37.340392+0000","last_fullsized":"2024-09-17T03:15:37.340392+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.347002+0000","last_change":"2024-09-17T03:15:37.347002+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.347002+0000","last_undegraded":"2024-09-17T03:15:37.347002+0000","last_fullsized":"2024-09-17T03:15:37.347002+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.343266+0000","last_change":"2024-09-17T03:15:37.343266+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.343266+0000","last_undegraded":"2024-09-17T03:15:37.343266+0000","last_fullsized":"2024-09-17T03:15:37.343266+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.380294+0000","last_change":"2024-09-17T03:15:39.374972+0000","last_active":"2024-09-17T03:15:39.380294+0000","last_peered":"2024-09-17T03:15:39.380294+0000","last_clean":"2024-09-17T03:15:39.380294+0000","last_became_active":"2024-09-17T03:15:39.374418+0000","last_became_peered":"2024-09-17T03:15:39.374418+0000","last_unstale":"2024-09-17T03:15:39.380294+0000","last_undegraded":"2024-09-17T03:15:39.380294+0000","last_fullsized":"2024-09-17T03:15:39.380294+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:53:49.563161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.340186+0000","last_change":"2024-09-17T03:15:37.340186+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.340186+0000","last_undegraded":"2024-09-17T03:15:37.340186+0000","last_fullsized":"2024-09-17T03:15:37.340186+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.340193+0000","last_change":"2024-09-17T03:15:37.340193+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.340193+0000","last_undegraded":"2024-09-17T03:15:37.340193+0000","last_fullsized":"2024-09-17T03:15:37.340193+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.340448+0000","last_change":"2024-09-17T03:15:37.340448+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.340448+0000","last_undegraded":"2024-09-17T03:15:37.340448+0000","last_fullsized":"2024-09-17T03:15:37.340448+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.348521+0000","last_change":"2024-09-17T03:15:37.348521+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.348521+0000","last_undegraded":"2024-09-17T03:15:37.348521+0000","last_fullsized":"2024-09-17T03:15:37.348521+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.342484+0000","last_change":"2024-09-17T03:15:37.342484+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.342484+0000","last_undegraded":"2024-09-17T03:15:37.342484+0000","last_fullsized":"2024-09-17T03:15:37.342484+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.345700+0000","last_change":"2024-09-17T03:15:37.345700+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.345700+0000","last_undegraded":"2024-09-17T03:15:37.345700+0000","last_fullsized":"2024-09-17T03:15:37.345700+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.357849+0000","last_change":"2024-09-17T03:15:38.361994+0000","last_active":"2024-09-17T03:15:39.357849+0000","last_peered":"2024-09-17T03:15:39.357849+0000","last_clean":"2024-09-17T03:15:39.357849+0000","last_became_active":"2024-09-17T03:15:38.361661+0000","last_became_peered":"2024-09-17T03:15:38.361661+0000","last_unstale":"2024-09-17T03:15:39.357849+0000","last_undegraded":"2024-09-17T03:15:39.357849+0000","last_fullsized":"2024-09-17T03:15:39.357849+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:46:31.452976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.341965+0000","last_change":"2024-09-17T03:15:37.341965+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.341965+0000","last_undegraded":"2024-09-17T03:15:37.341965+0000","last_fullsized":"2024-09-17T03:15:37.341965+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.342669+0000","last_change":"2024-09-17T03:15:37.342669+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.342669+0000","last_undegraded":"2024-09-17T03:15:37.342669+0000","last_fullsized":"2024-09-17T03:15:37.342669+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.347427+0000","last_change":"2024-09-17T03:15:37.347427+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.347427+0000","last_undegraded":"2024-09-17T03:15:37.347427+0000","last_fullsized":"2024-09-17T03:15:37.347427+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.347233+0000","last_change":"2024-09-17T03:15:37.347233+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.347233+0000","last_undegraded":"2024-09-17T03:15:37.347233+0000","last_fullsized":"2024-09-17T03:15:37.347233+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.375247+0000","last_change":"2024-09-17T03:15:39.366303+0000","last_active":"2024-09-17T03:15:39.375247+0000","last_peered":"2024-09-17T03:15:39.375247+0000","last_clean":"2024-09-17T03:15:39.375247+0000","last_became_active":"2024-09-17T03:15:39.365953+0000","last_became_peered":"2024-09-17T03:15:39.365953+0000","last_unstale":"2024-09-17T03:15:39.375247+0000","last_undegraded":"2024-09-17T03:15:39.375247+0000","last_fullsized":"2024-09-17T03:15:39.375247+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:51.052810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.356414+0000","last_change":"2024-09-17T03:15:38.362902+0000","last_active":"2024-09-17T03:15:39.356414+0000","last_peered":"2024-09-17T03:15:39.356414+0000","last_clean":"2024-09-17T03:15:39.356414+0000","last_became_active":"2024-09-17T03:15:38.362598+0000","last_became_peered":"2024-09-17T03:15:38.362598+0000","last_unstale":"2024-09-17T03:15:39.356414+0000","last_undegraded":"2024-09-17T03:15:39.356414+0000","last_fullsized":"2024-09-17T03:15:39.356414+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:56.538985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.341949+0000","last_change":"2024-09-17T03:15:37.341949+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.341949+0000","last_undegraded":"2024-09-17T03:15:37.341949+0000","last_fullsized":"2024-09-17T03:15:37.341949+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.358479+0000","last_change":"2024-09-17T03:15:38.361181+0000","last_active":"2024-09-17T03:15:39.358479+0000","last_peered":"2024-09-17T03:15:39.358479+0000","last_clean":"2024-09-17T03:15:39.358479+0000","last_became_active":"2024-09-17T03:15:38.360468+0000","last_became_peered":"2024-09-17T03:15:38.360468+0000","last_unstale":"2024-09-17T03:15:39.358479+0000","last_undegraded":"2024-09-17T03:15:39.358479+0000","last_fullsized":"2024-09-17T03:15:39.358479+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:52.635369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.342475+0000","last_change":"2024-09-17T03:15:37.342475+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.342475+0000","last_undegraded":"2024-09-17T03:15:37.342475+0000","last_fullsized":"2024-09-17T03:15:37.342475+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.352546+0000","last_change":"2024-09-17T03:15:37.352546+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.352546+0000","last_undegraded":"2024-09-17T03:15:37.352546+0000","last_fullsized":"2024-09-17T03:15:37.352546+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.343143+0000","last_change":"2024-09-17T03:15:37.343143+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.343143+0000","last_undegraded":"2024-09-17T03:15:37.343143+0000","last_fullsized":"2024-09-17T03:15:37.343143+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":153,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T03:15:37.353340+0000","last_change":"2024-09-17T03:14:04.862167+0000","last_active":"2024-09-17T03:15:37.353340+0000","last_peered":"2024-09-17T03:15:37.353340+0000","last_clean":"2024-09-17T03:15:37.353340+0000","last_became_active":"2024-09-17T03:14:04.861731+0000","last_became_peered":"2024-09-17T03:14:04.861731+0000","last_unstale":"2024-09-17T03:15:37.353340+0000","last_undegraded":"2024-09-17T03:15:37.353340+0000","last_fullsized":"2024-09-17T03:15:37.353340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_clean_scrub_stamp":"2024-09-17T03:14:03.847721+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:32.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":1},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":536576,"data_stored":215463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":536576,"data_stored":215463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509463,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1142784,"data_stored":811648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":536576,"data_stored":215463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":536576,"data_stored":215463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509464,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28140,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343700,"statfs":{"total":96636764160,"available":96607948800,"internally_reserved":0,"allocated":1130496,"data_stored":805831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":536576,"data_stored":215463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":561152,"data_stored":227555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:15:42.714 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:15:42.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.862+0000 7f7cc7112640 1 -- 172.21.15.87:0/628415411 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc00d2da0 msgr2=0x7f7cc00d31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:42.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.862+0000 7f7cc7112640 1 --2- 172.21.15.87:0/628415411 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc00d2da0 0x7f7cc00d31b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0005fd0 tx=0x7f7cb002a4a0 comp rx=0 tx=0).stop 2024-09-17T03:15:42.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.862+0000 7f7cc7112640 1 -- 172.21.15.87:0/628415411 shutdown_connections 2024-09-17T03:15:42.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.862+0000 7f7cc7112640 1 --2- 172.21.15.87:0/628415411 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7cc00b5ea0 0x7f7cc00b62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:42.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.862+0000 7f7cc7112640 1 --2- 172.21.15.87:0/628415411 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc00d2da0 0x7f7cc00d31b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:42.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.862+0000 7f7cc7112640 1 -- 172.21.15.87:0/628415411 >> 172.21.15.87:0/628415411 conn(0x7f7cc00698b0 msgr2=0x7f7cc00d4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:42.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.862+0000 7f7cc7112640 1 -- 172.21.15.87:0/628415411 shutdown_connections 2024-09-17T03:15:42.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.862+0000 7f7cc7112640 1 -- 172.21.15.87:0/628415411 wait complete. 2024-09-17T03:15:42.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.864+0000 7f7cc7112640 1 Processor -- start 2024-09-17T03:15:42.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.864+0000 7f7cc7112640 1 -- start start 2024-09-17T03:15:42.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.865+0000 7f7cc7112640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc00b5ea0 0x7f7cc01f8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:42.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.865+0000 7f7cc7112640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cc00cf140 0x7f7cc01f6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:42.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.865+0000 7f7cc7112640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7cc00d2da0 0x7f7cc00ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:42.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.865+0000 7f7cc7112640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc01621e0 con 0x7f7cc00b5ea0 2024-09-17T03:15:42.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.865+0000 7f7cc7112640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc00b9630 con 0x7f7cc00cf140 2024-09-17T03:15:42.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.865+0000 7f7cc7112640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f7cc00cfc60 con 0x7f7cc00d2da0 2024-09-17T03:15:42.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.865+0000 7f7cc4eae640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc00b5ea0 0x7f7cc01f8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:42.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.865+0000 7f7cc4eae640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc00b5ea0 0x7f7cc01f8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36134/0 (socket says 172.21.15.87:36134) 2024-09-17T03:15:42.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.865+0000 7f7cc4eae640 1 -- 172.21.15.87:0/2725370379 learned_addr learned my addr 172.21.15.87:0/2725370379 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:42.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.865+0000 7f7cc56af640 1 --2- 172.21.15.87:0/2725370379 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7cc00d2da0 0x7f7cc00ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:42.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.865+0000 7f7cbffff640 1 --2- 172.21.15.87:0/2725370379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cc00cf140 0x7f7cc01f6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:42.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.866+0000 7f7cc4eae640 1 -- 172.21.15.87:0/2725370379 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7cc00d2da0 msgr2=0x7f7cc00ca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:42.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.866+0000 7f7cc4eae640 1 --2- 172.21.15.87:0/2725370379 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7cc00d2da0 0x7f7cc00ca6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:42.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.866+0000 7f7cc4eae640 1 -- 172.21.15.87:0/2725370379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cc00cf140 msgr2=0x7f7cc01f6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:42.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.866+0000 7f7cc4eae640 1 --2- 172.21.15.87:0/2725370379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cc00cf140 0x7f7cc01f6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:42.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.866+0000 7f7cc4eae640 1 -- 172.21.15.87:0/2725370379 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc01f6c80 con 0x7f7cc00b5ea0 2024-09-17T03:15:42.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.866+0000 7f7cc4eae640 1 --2- 172.21.15.87:0/2725370379 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc00b5ea0 0x7f7cc01f8cd0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0034040 tx=0x7f7cb0003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:42.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.866+0000 7f7cbd7fa640 1 -- 172.21.15.87:0/2725370379 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb003e030 con 0x7f7cc00b5ea0 2024-09-17T03:15:42.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.866+0000 7f7cbd7fa640 1 -- 172.21.15.87:0/2725370379 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cb0003e00 con 0x7f7cc00b5ea0 2024-09-17T03:15:42.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.866+0000 7f7cbd7fa640 1 -- 172.21.15.87:0/2725370379 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0035040 con 0x7f7cc00b5ea0 2024-09-17T03:15:42.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.866+0000 7f7cc7112640 1 -- 172.21.15.87:0/2725370379 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc00ab220 con 0x7f7cc00b5ea0 2024-09-17T03:15:42.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.866+0000 7f7cc7112640 1 -- 172.21.15.87:0/2725370379 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc00ab7e0 con 0x7f7cc00b5ea0 2024-09-17T03:15:42.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.867+0000 7f7cc7112640 1 -- 172.21.15.87:0/2725370379 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cc00b62b0 con 0x7f7cc00b5ea0 2024-09-17T03:15:42.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.868+0000 7f7cbd7fa640 1 -- 172.21.15.87:0/2725370379 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7cb0041030 con 0x7f7cc00b5ea0 2024-09-17T03:15:42.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.868+0000 7f7cbd7fa640 1 --2- 172.21.15.87:0/2725370379 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7cac078b90 0x7f7cac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:42.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.869+0000 7f7cbd7fa640 1 -- 172.21.15.87:0/2725370379 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f7cb00bb410 con 0x7f7cc00b5ea0 2024-09-17T03:15:42.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.869+0000 7f7cbffff640 1 --2- 172.21.15.87:0/2725370379 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7cac078b90 0x7f7cac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:42.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.869+0000 7f7cbffff640 1 --2- 172.21.15.87:0/2725370379 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7cac078b90 0x7f7cac07b0a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7cc01f38b0 tx=0x7f7cb400b2e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:42.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:42.873+0000 7f7cbd7fa640 1 -- 172.21.15.87:0/2725370379 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7cc01f6c80 con 0x7f7cc00b5ea0 2024-09-17T03:15:43.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.045+0000 7f7cc7112640 1 -- 172.21.15.87:0/2725370379 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7cc00aad60 con 0x7f7cac078b90 2024-09-17T03:15:43.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.049+0000 7f7cbd7fa640 1 -- 172.21.15.87:0/2725370379 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241868 (secure 0 0 0) 0x7f7cb4010b40 con 0x7f7cac078b90 2024-09-17T03:15:43.051 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:43.052 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-09-17T03:15:43.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.052+0000 7f7cc7112640 1 -- 172.21.15.87:0/2725370379 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7cac078b90 msgr2=0x7f7cac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:43.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.052+0000 7f7cc7112640 1 --2- 172.21.15.87:0/2725370379 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7cac078b90 0x7f7cac07b0a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7cc01f38b0 tx=0x7f7cb400b2e0 comp rx=0 tx=0).stop 2024-09-17T03:15:43.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.052+0000 7f7cc7112640 1 -- 172.21.15.87:0/2725370379 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc00b5ea0 msgr2=0x7f7cc01f8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:43.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.052+0000 7f7cc7112640 1 --2- 172.21.15.87:0/2725370379 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc00b5ea0 0x7f7cc01f8cd0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0034040 tx=0x7f7cb0003910 comp rx=0 tx=0).stop 2024-09-17T03:15:43.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.052+0000 7f7cc7112640 1 -- 172.21.15.87:0/2725370379 shutdown_connections 2024-09-17T03:15:43.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.052+0000 7f7cc7112640 1 --2- 172.21.15.87:0/2725370379 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7cac078b90 0x7f7cac07b0a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:43.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.052+0000 7f7cc7112640 1 --2- 172.21.15.87:0/2725370379 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7cc00d2da0 0x7f7cc00ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:43.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.052+0000 7f7cc7112640 1 --2- 172.21.15.87:0/2725370379 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cc00cf140 0x7f7cc01f6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:43.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.052+0000 7f7cc7112640 1 --2- 172.21.15.87:0/2725370379 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc00b5ea0 0x7f7cc01f8cd0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:43.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.052+0000 7f7cc7112640 1 -- 172.21.15.87:0/2725370379 >> 172.21.15.87:0/2725370379 conn(0x7f7cc00698b0 msgr2=0x7f7cc015f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:43.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.052+0000 7f7cc7112640 1 -- 172.21.15.87:0/2725370379 shutdown_connections 2024-09-17T03:15:43.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.052+0000 7f7cc7112640 1 -- 172.21.15.87:0/2725370379 wait complete. 2024-09-17T03:15:43.062 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":94,"stamp":"2024-09-17T03:15:41.438402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":86,"acting":86,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":35,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229972,"kb_used_data":5388,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744748,"statfs":{"total":773094113280,"available":772858621952,"internally_reserved":0,"allocated":5517312,"data_stored":2922349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.099408"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.377499+0000","last_change":"2024-09-17T03:15:39.368117+0000","last_active":"2024-09-17T03:15:39.377499+0000","last_peered":"2024-09-17T03:15:39.377499+0000","last_clean":"2024-09-17T03:15:39.377499+0000","last_became_active":"2024-09-17T03:15:39.366527+0000","last_became_peered":"2024-09-17T03:15:39.366527+0000","last_unstale":"2024-09-17T03:15:39.377499+0000","last_undegraded":"2024-09-17T03:15:39.377499+0000","last_fullsized":"2024-09-17T03:15:39.377499+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:00:48.160980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.377520+0000","last_change":"2024-09-17T03:15:39.367252+0000","last_active":"2024-09-17T03:15:39.377520+0000","last_peered":"2024-09-17T03:15:39.377520+0000","last_clean":"2024-09-17T03:15:39.377520+0000","last_became_active":"2024-09-17T03:15:39.366982+0000","last_became_peered":"2024-09-17T03:15:39.366982+0000","last_unstale":"2024-09-17T03:15:39.377520+0000","last_undegraded":"2024-09-17T03:15:39.377520+0000","last_fullsized":"2024-09-17T03:15:39.377520+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:02:23.197700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.369870+0000","last_change":"2024-09-17T03:15:39.366135+0000","last_active":"2024-09-17T03:15:39.369870+0000","last_peered":"2024-09-17T03:15:39.369870+0000","last_clean":"2024-09-17T03:15:39.369870+0000","last_became_active":"2024-09-17T03:15:39.365800+0000","last_became_peered":"2024-09-17T03:15:39.365800+0000","last_unstale":"2024-09-17T03:15:39.369870+0000","last_undegraded":"2024-09-17T03:15:39.369870+0000","last_fullsized":"2024-09-17T03:15:39.369870+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:17:16.473419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.380167+0000","last_change":"2024-09-17T03:15:39.371434+0000","last_active":"2024-09-17T03:15:39.380167+0000","last_peered":"2024-09-17T03:15:39.380167+0000","last_clean":"2024-09-17T03:15:39.380167+0000","last_became_active":"2024-09-17T03:15:39.371098+0000","last_became_peered":"2024-09-17T03:15:39.371098+0000","last_unstale":"2024-09-17T03:15:39.380167+0000","last_undegraded":"2024-09-17T03:15:39.380167+0000","last_fullsized":"2024-09-17T03:15:39.380167+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:33.488929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.380327+0000","last_change":"2024-09-17T03:15:39.375041+0000","last_active":"2024-09-17T03:15:39.380327+0000","last_peered":"2024-09-17T03:15:39.380327+0000","last_clean":"2024-09-17T03:15:39.380327+0000","last_became_active":"2024-09-17T03:15:39.374237+0000","last_became_peered":"2024-09-17T03:15:39.374237+0000","last_unstale":"2024-09-17T03:15:39.380327+0000","last_undegraded":"2024-09-17T03:15:39.380327+0000","last_fullsized":"2024-09-17T03:15:39.380327+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:06.672069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.377585+0000","last_change":"2024-09-17T03:15:39.368629+0000","last_active":"2024-09-17T03:15:39.377585+0000","last_peered":"2024-09-17T03:15:39.377585+0000","last_clean":"2024-09-17T03:15:39.377585+0000","last_became_active":"2024-09-17T03:15:39.368078+0000","last_became_peered":"2024-09-17T03:15:39.368078+0000","last_unstale":"2024-09-17T03:15:39.377585+0000","last_undegraded":"2024-09-17T03:15:39.377585+0000","last_fullsized":"2024-09-17T03:15:39.377585+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:24.126150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.380304+0000","last_change":"2024-09-17T03:15:39.374436+0000","last_active":"2024-09-17T03:15:39.380304+0000","last_peered":"2024-09-17T03:15:39.380304+0000","last_clean":"2024-09-17T03:15:39.380304+0000","last_became_active":"2024-09-17T03:15:39.372074+0000","last_became_peered":"2024-09-17T03:15:39.372074+0000","last_unstale":"2024-09-17T03:15:39.380304+0000","last_undegraded":"2024-09-17T03:15:39.380304+0000","last_fullsized":"2024-09-17T03:15:39.380304+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:48:37.306791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.377548+0000","last_change":"2024-09-17T03:15:39.367577+0000","last_active":"2024-09-17T03:15:39.377548+0000","last_peered":"2024-09-17T03:15:39.377548+0000","last_clean":"2024-09-17T03:15:39.377548+0000","last_became_active":"2024-09-17T03:15:39.367310+0000","last_became_peered":"2024-09-17T03:15:39.367310+0000","last_unstale":"2024-09-17T03:15:39.377548+0000","last_undegraded":"2024-09-17T03:15:39.377548+0000","last_fullsized":"2024-09-17T03:15:39.377548+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:57:05.939726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.347030+0000","last_change":"2024-09-17T03:15:37.347030+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.347030+0000","last_undegraded":"2024-09-17T03:15:37.347030+0000","last_fullsized":"2024-09-17T03:15:37.347030+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.344765+0000","last_change":"2024-09-17T03:15:37.344765+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.344765+0000","last_undegraded":"2024-09-17T03:15:37.344765+0000","last_fullsized":"2024-09-17T03:15:37.344765+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.342115+0000","last_change":"2024-09-17T03:15:37.342115+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.342115+0000","last_undegraded":"2024-09-17T03:15:37.342115+0000","last_fullsized":"2024-09-17T03:15:37.342115+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.343314+0000","last_change":"2024-09-17T03:15:37.343314+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.343314+0000","last_undegraded":"2024-09-17T03:15:37.343314+0000","last_fullsized":"2024-09-17T03:15:37.343314+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"49'1","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.384912+0000","last_change":"2024-09-17T03:15:38.358694+0000","last_active":"2024-09-17T03:15:39.384912+0000","last_peered":"2024-09-17T03:15:39.384912+0000","last_clean":"2024-09-17T03:15:39.384912+0000","last_became_active":"2024-09-17T03:15:38.358307+0000","last_became_peered":"2024-09-17T03:15:38.358307+0000","last_unstale":"2024-09-17T03:15:39.384912+0000","last_undegraded":"2024-09-17T03:15:39.384912+0000","last_fullsized":"2024-09-17T03:15:39.384912+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:15.429280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.347253+0000","last_change":"2024-09-17T03:15:37.347253+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.347253+0000","last_undegraded":"2024-09-17T03:15:37.347253+0000","last_fullsized":"2024-09-17T03:15:37.347253+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.347297+0000","last_change":"2024-09-17T03:15:37.347297+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.347297+0000","last_undegraded":"2024-09-17T03:15:37.347297+0000","last_fullsized":"2024-09-17T03:15:37.347297+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.353431+0000","last_change":"2024-09-17T03:15:38.362750+0000","last_active":"2024-09-17T03:15:39.353431+0000","last_peered":"2024-09-17T03:15:39.353431+0000","last_clean":"2024-09-17T03:15:39.353431+0000","last_became_active":"2024-09-17T03:15:38.362412+0000","last_became_peered":"2024-09-17T03:15:38.362412+0000","last_unstale":"2024-09-17T03:15:39.353431+0000","last_undegraded":"2024-09-17T03:15:39.353431+0000","last_fullsized":"2024-09-17T03:15:39.353431+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:40:22.976291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.348269+0000","last_change":"2024-09-17T03:15:37.348269+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.348269+0000","last_undegraded":"2024-09-17T03:15:37.348269+0000","last_fullsized":"2024-09-17T03:15:37.348269+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.348206+0000","last_change":"2024-09-17T03:15:37.348206+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.348206+0000","last_undegraded":"2024-09-17T03:15:37.348206+0000","last_fullsized":"2024-09-17T03:15:37.348206+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.347947+0000","last_change":"2024-09-17T03:15:37.347947+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.347947+0000","last_undegraded":"2024-09-17T03:15:37.347947+0000","last_fullsized":"2024-09-17T03:15:37.347947+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.342227+0000","last_change":"2024-09-17T03:15:37.342227+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.342227+0000","last_undegraded":"2024-09-17T03:15:37.342227+0000","last_fullsized":"2024-09-17T03:15:37.342227+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.340392+0000","last_change":"2024-09-17T03:15:37.340392+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.340392+0000","last_undegraded":"2024-09-17T03:15:37.340392+0000","last_fullsized":"2024-09-17T03:15:37.340392+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.347002+0000","last_change":"2024-09-17T03:15:37.347002+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.347002+0000","last_undegraded":"2024-09-17T03:15:37.347002+0000","last_fullsized":"2024-09-17T03:15:37.347002+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.343266+0000","last_change":"2024-09-17T03:15:37.343266+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.343266+0000","last_undegraded":"2024-09-17T03:15:37.343266+0000","last_fullsized":"2024-09-17T03:15:37.343266+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.380294+0000","last_change":"2024-09-17T03:15:39.374972+0000","last_active":"2024-09-17T03:15:39.380294+0000","last_peered":"2024-09-17T03:15:39.380294+0000","last_clean":"2024-09-17T03:15:39.380294+0000","last_became_active":"2024-09-17T03:15:39.374418+0000","last_became_peered":"2024-09-17T03:15:39.374418+0000","last_unstale":"2024-09-17T03:15:39.380294+0000","last_undegraded":"2024-09-17T03:15:39.380294+0000","last_fullsized":"2024-09-17T03:15:39.380294+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:53:49.563161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.340186+0000","last_change":"2024-09-17T03:15:37.340186+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.340186+0000","last_undegraded":"2024-09-17T03:15:37.340186+0000","last_fullsized":"2024-09-17T03:15:37.340186+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.340193+0000","last_change":"2024-09-17T03:15:37.340193+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.340193+0000","last_undegraded":"2024-09-17T03:15:37.340193+0000","last_fullsized":"2024-09-17T03:15:37.340193+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.340448+0000","last_change":"2024-09-17T03:15:37.340448+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.340448+0000","last_undegraded":"2024-09-17T03:15:37.340448+0000","last_fullsized":"2024-09-17T03:15:37.340448+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.348521+0000","last_change":"2024-09-17T03:15:37.348521+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.348521+0000","last_undegraded":"2024-09-17T03:15:37.348521+0000","last_fullsized":"2024-09-17T03:15:37.348521+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.342484+0000","last_change":"2024-09-17T03:15:37.342484+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.342484+0000","last_undegraded":"2024-09-17T03:15:37.342484+0000","last_fullsized":"2024-09-17T03:15:37.342484+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.345700+0000","last_change":"2024-09-17T03:15:37.345700+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.345700+0000","last_undegraded":"2024-09-17T03:15:37.345700+0000","last_fullsized":"2024-09-17T03:15:37.345700+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.357849+0000","last_change":"2024-09-17T03:15:38.361994+0000","last_active":"2024-09-17T03:15:39.357849+0000","last_peered":"2024-09-17T03:15:39.357849+0000","last_clean":"2024-09-17T03:15:39.357849+0000","last_became_active":"2024-09-17T03:15:38.361661+0000","last_became_peered":"2024-09-17T03:15:38.361661+0000","last_unstale":"2024-09-17T03:15:39.357849+0000","last_undegraded":"2024-09-17T03:15:39.357849+0000","last_fullsized":"2024-09-17T03:15:39.357849+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:46:31.452976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.341965+0000","last_change":"2024-09-17T03:15:37.341965+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.341965+0000","last_undegraded":"2024-09-17T03:15:37.341965+0000","last_fullsized":"2024-09-17T03:15:37.341965+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.342669+0000","last_change":"2024-09-17T03:15:37.342669+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.342669+0000","last_undegraded":"2024-09-17T03:15:37.342669+0000","last_fullsized":"2024-09-17T03:15:37.342669+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.347427+0000","last_change":"2024-09-17T03:15:37.347427+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.347427+0000","last_undegraded":"2024-09-17T03:15:37.347427+0000","last_fullsized":"2024-09-17T03:15:37.347427+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.347233+0000","last_change":"2024-09-17T03:15:37.347233+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.347233+0000","last_undegraded":"2024-09-17T03:15:37.347233+0000","last_fullsized":"2024-09-17T03:15:37.347233+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.375247+0000","last_change":"2024-09-17T03:15:39.366303+0000","last_active":"2024-09-17T03:15:39.375247+0000","last_peered":"2024-09-17T03:15:39.375247+0000","last_clean":"2024-09-17T03:15:39.375247+0000","last_became_active":"2024-09-17T03:15:39.365953+0000","last_became_peered":"2024-09-17T03:15:39.365953+0000","last_unstale":"2024-09-17T03:15:39.375247+0000","last_undegraded":"2024-09-17T03:15:39.375247+0000","last_fullsized":"2024-09-17T03:15:39.375247+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:51.052810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.356414+0000","last_change":"2024-09-17T03:15:38.362902+0000","last_active":"2024-09-17T03:15:39.356414+0000","last_peered":"2024-09-17T03:15:39.356414+0000","last_clean":"2024-09-17T03:15:39.356414+0000","last_became_active":"2024-09-17T03:15:38.362598+0000","last_became_peered":"2024-09-17T03:15:38.362598+0000","last_unstale":"2024-09-17T03:15:39.356414+0000","last_undegraded":"2024-09-17T03:15:39.356414+0000","last_fullsized":"2024-09-17T03:15:39.356414+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:56.538985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.341949+0000","last_change":"2024-09-17T03:15:37.341949+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.341949+0000","last_undegraded":"2024-09-17T03:15:37.341949+0000","last_fullsized":"2024-09-17T03:15:37.341949+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.358479+0000","last_change":"2024-09-17T03:15:38.361181+0000","last_active":"2024-09-17T03:15:39.358479+0000","last_peered":"2024-09-17T03:15:39.358479+0000","last_clean":"2024-09-17T03:15:39.358479+0000","last_became_active":"2024-09-17T03:15:38.360468+0000","last_became_peered":"2024-09-17T03:15:38.360468+0000","last_unstale":"2024-09-17T03:15:39.358479+0000","last_undegraded":"2024-09-17T03:15:39.358479+0000","last_fullsized":"2024-09-17T03:15:39.358479+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:52.635369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.342475+0000","last_change":"2024-09-17T03:15:37.342475+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.342475+0000","last_undegraded":"2024-09-17T03:15:37.342475+0000","last_fullsized":"2024-09-17T03:15:37.342475+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.352546+0000","last_change":"2024-09-17T03:15:37.352546+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.352546+0000","last_undegraded":"2024-09-17T03:15:37.352546+0000","last_fullsized":"2024-09-17T03:15:37.352546+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-17T03:15:38.335805+0000","last_change":"2024-09-17T03:15:38.335805+0000","last_active":"2024-09-17T03:15:38.335805+0000","last_peered":"2024-09-17T03:15:38.335805+0000","last_clean":"2024-09-17T03:15:38.335805+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:38.335805+0000","last_undegraded":"2024-09-17T03:15:38.335805+0000","last_fullsized":"2024-09-17T03:15:38.335805+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2024-09-17T03:15:37.343143+0000","last_change":"2024-09-17T03:15:37.343143+0000","last_active":"2024-09-17T03:15:37.333291+0000","last_peered":"2024-09-17T03:15:37.333291+0000","last_clean":"2024-09-17T03:15:37.333291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-17T03:15:37.343143+0000","last_undegraded":"2024-09-17T03:15:37.343143+0000","last_fullsized":"2024-09-17T03:15:37.343143+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":153,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-17T03:15:37.353340+0000","last_change":"2024-09-17T03:14:04.862167+0000","last_active":"2024-09-17T03:15:37.353340+0000","last_peered":"2024-09-17T03:15:37.353340+0000","last_clean":"2024-09-17T03:15:37.353340+0000","last_became_active":"2024-09-17T03:14:04.861731+0000","last_became_peered":"2024-09-17T03:14:04.861731+0000","last_unstale":"2024-09-17T03:15:37.353340+0000","last_undegraded":"2024-09-17T03:15:37.353340+0000","last_fullsized":"2024-09-17T03:15:37.353340+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_clean_scrub_stamp":"2024-09-17T03:14:03.847721+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:32.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":1},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":536576,"data_stored":215463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":536576,"data_stored":215463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509463,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32304,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339536,"statfs":{"total":96636764160,"available":96603684864,"internally_reserved":0,"allocated":1142784,"data_stored":811648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":536576,"data_stored":215463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509464,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":536576,"data_stored":215463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509464,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28140,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343700,"statfs":{"total":96636764160,"available":96607948800,"internally_reserved":0,"allocated":1130496,"data_stored":805831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27556,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344284,"statfs":{"total":96636764160,"available":96608546816,"internally_reserved":0,"allocated":536576,"data_stored":215463,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":561152,"data_stored":227555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:15:43.066 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3f is not active+clean 2024-09-17T03:15:43.066 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.066 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3e is not active+clean 2024-09-17T03:15:43.067 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.067 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3c is not active+clean 2024-09-17T03:15:43.067 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.067 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3b is not active+clean 2024-09-17T03:15:43.067 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.067 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3a is not active+clean 2024-09-17T03:15:43.067 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.067 INFO:tasks.ceph.ceph_manager.ceph:PG 8.39 is not active+clean 2024-09-17T03:15:43.067 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.067 INFO:tasks.ceph.ceph_manager.ceph:PG 8.38 is not active+clean 2024-09-17T03:15:43.068 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.068 INFO:tasks.ceph.ceph_manager.ceph:PG 8.36 is not active+clean 2024-09-17T03:15:43.068 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.068 INFO:tasks.ceph.ceph_manager.ceph:PG 8.35 is not active+clean 2024-09-17T03:15:43.068 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.068 INFO:tasks.ceph.ceph_manager.ceph:PG 8.34 is not active+clean 2024-09-17T03:15:43.068 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.068 INFO:tasks.ceph.ceph_manager.ceph:PG 8.33 is not active+clean 2024-09-17T03:15:43.068 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.068 INFO:tasks.ceph.ceph_manager.ceph:PG 8.32 is not active+clean 2024-09-17T03:15:43.068 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.069 INFO:tasks.ceph.ceph_manager.ceph:PG 8.31 is not active+clean 2024-09-17T03:15:43.069 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.069 INFO:tasks.ceph.ceph_manager.ceph:PG 8.30 is not active+clean 2024-09-17T03:15:43.069 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.069 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2f is not active+clean 2024-09-17T03:15:43.069 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.069 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2e is not active+clean 2024-09-17T03:15:43.069 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.069 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2c is not active+clean 2024-09-17T03:15:43.069 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.070 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2b is not active+clean 2024-09-17T03:15:43.070 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.070 INFO:tasks.ceph.ceph_manager.ceph:PG 8.27 is not active+clean 2024-09-17T03:15:43.070 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.070 INFO:tasks.ceph.ceph_manager.ceph:PG 8.25 is not active+clean 2024-09-17T03:15:43.070 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.070 INFO:tasks.ceph.ceph_manager.ceph:PG 8.24 is not active+clean 2024-09-17T03:15:43.071 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.071 INFO:tasks.ceph.ceph_manager.ceph:PG 8.23 is not active+clean 2024-09-17T03:15:43.071 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.071 INFO:tasks.ceph.ceph_manager.ceph:PG 8.22 is not active+clean 2024-09-17T03:15:43.071 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.071 INFO:tasks.ceph.ceph_manager.ceph:PG 8.20 is not active+clean 2024-09-17T03:15:43.071 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.072 INFO:tasks.ceph.ceph_manager.ceph:PG 8.10 is not active+clean 2024-09-17T03:15:43.072 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.072 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1f is not active+clean 2024-09-17T03:15:43.072 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.347030+0000', 'last_change': '2024-09-17T03:15:37.347030+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.347030+0000', 'last_undegraded': '2024-09-17T03:15:37.347030+0000', 'last_fullsized': '2024-09-17T03:15:37.347030+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-17T03:15:43.072 INFO:tasks.ceph.ceph_manager.ceph:PG 8.11 is not active+clean 2024-09-17T03:15:43.072 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.072 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1e is not active+clean 2024-09-17T03:15:43.072 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.344765+0000', 'last_change': '2024-09-17T03:15:37.344765+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.344765+0000', 'last_undegraded': '2024-09-17T03:15:37.344765+0000', 'last_fullsized': '2024-09-17T03:15:37.344765+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T03:15:43.072 INFO:tasks.ceph.ceph_manager.ceph:PG 8.12 is not active+clean 2024-09-17T03:15:43.072 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.073 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1d is not active+clean 2024-09-17T03:15:43.073 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.342115+0000', 'last_change': '2024-09-17T03:15:37.342115+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.342115+0000', 'last_undegraded': '2024-09-17T03:15:37.342115+0000', 'last_fullsized': '2024-09-17T03:15:37.342115+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:15:43.073 INFO:tasks.ceph.ceph_manager.ceph:PG 8.13 is not active+clean 2024-09-17T03:15:43.073 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.073 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1c is not active+clean 2024-09-17T03:15:43.073 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.343314+0000', 'last_change': '2024-09-17T03:15:37.343314+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.343314+0000', 'last_undegraded': '2024-09-17T03:15:37.343314+0000', 'last_fullsized': '2024-09-17T03:15:37.343314+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:15:43.073 INFO:tasks.ceph.ceph_manager.ceph:PG 8.14 is not active+clean 2024-09-17T03:15:43.073 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.073 INFO:tasks.ceph.ceph_manager.ceph:PG 8.15 is not active+clean 2024-09-17T03:15:43.074 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.074 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1a is not active+clean 2024-09-17T03:15:43.074 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.347253+0000', 'last_change': '2024-09-17T03:15:37.347253+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.347253+0000', 'last_undegraded': '2024-09-17T03:15:37.347253+0000', 'last_fullsized': '2024-09-17T03:15:37.347253+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T03:15:43.074 INFO:tasks.ceph.ceph_manager.ceph:PG 8.16 is not active+clean 2024-09-17T03:15:43.074 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.074 INFO:tasks.ceph.ceph_manager.ceph:PG 7.19 is not active+clean 2024-09-17T03:15:43.074 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.347297+0000', 'last_change': '2024-09-17T03:15:37.347297+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.347297+0000', 'last_undegraded': '2024-09-17T03:15:37.347297+0000', 'last_fullsized': '2024-09-17T03:15:37.347297+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T03:15:43.075 INFO:tasks.ceph.ceph_manager.ceph:PG 8.17 is not active+clean 2024-09-17T03:15:43.075 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.075 INFO:tasks.ceph.ceph_manager.ceph:PG 8.18 is not active+clean 2024-09-17T03:15:43.075 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.075 INFO:tasks.ceph.ceph_manager.ceph:PG 7.17 is not active+clean 2024-09-17T03:15:43.075 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.348269+0000', 'last_change': '2024-09-17T03:15:37.348269+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.348269+0000', 'last_undegraded': '2024-09-17T03:15:37.348269+0000', 'last_fullsized': '2024-09-17T03:15:37.348269+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T03:15:43.075 INFO:tasks.ceph.ceph_manager.ceph:PG 8.19 is not active+clean 2024-09-17T03:15:43.075 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.075 INFO:tasks.ceph.ceph_manager.ceph:PG 7.16 is not active+clean 2024-09-17T03:15:43.075 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.348206+0000', 'last_change': '2024-09-17T03:15:37.348206+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.348206+0000', 'last_undegraded': '2024-09-17T03:15:37.348206+0000', 'last_fullsized': '2024-09-17T03:15:37.348206+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T03:15:43.076 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1a is not active+clean 2024-09-17T03:15:43.076 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.076 INFO:tasks.ceph.ceph_manager.ceph:PG 7.15 is not active+clean 2024-09-17T03:15:43.076 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.347947+0000', 'last_change': '2024-09-17T03:15:37.347947+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.347947+0000', 'last_undegraded': '2024-09-17T03:15:37.347947+0000', 'last_fullsized': '2024-09-17T03:15:37.347947+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T03:15:43.076 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1b is not active+clean 2024-09-17T03:15:43.076 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.076 INFO:tasks.ceph.ceph_manager.ceph:PG 7.14 is not active+clean 2024-09-17T03:15:43.076 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.342227+0000', 'last_change': '2024-09-17T03:15:37.342227+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.342227+0000', 'last_undegraded': '2024-09-17T03:15:37.342227+0000', 'last_fullsized': '2024-09-17T03:15:37.342227+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:15:43.076 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1c is not active+clean 2024-09-17T03:15:43.076 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.077 INFO:tasks.ceph.ceph_manager.ceph:PG 7.13 is not active+clean 2024-09-17T03:15:43.077 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.340392+0000', 'last_change': '2024-09-17T03:15:37.340392+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.340392+0000', 'last_undegraded': '2024-09-17T03:15:37.340392+0000', 'last_fullsized': '2024-09-17T03:15:37.340392+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T03:15:43.077 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1d is not active+clean 2024-09-17T03:15:43.077 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.077 INFO:tasks.ceph.ceph_manager.ceph:PG 7.12 is not active+clean 2024-09-17T03:15:43.077 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.347002+0000', 'last_change': '2024-09-17T03:15:37.347002+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.347002+0000', 'last_undegraded': '2024-09-17T03:15:37.347002+0000', 'last_fullsized': '2024-09-17T03:15:37.347002+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T03:15:43.077 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1e is not active+clean 2024-09-17T03:15:43.077 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.078 INFO:tasks.ceph.ceph_manager.ceph:PG 7.11 is not active+clean 2024-09-17T03:15:43.078 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.343266+0000', 'last_change': '2024-09-17T03:15:37.343266+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.343266+0000', 'last_undegraded': '2024-09-17T03:15:37.343266+0000', 'last_fullsized': '2024-09-17T03:15:37.343266+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:15:43.078 INFO:tasks.ceph.ceph_manager.ceph:PG 7.10 is not active+clean 2024-09-17T03:15:43.078 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.340186+0000', 'last_change': '2024-09-17T03:15:37.340186+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.340186+0000', 'last_undegraded': '2024-09-17T03:15:37.340186+0000', 'last_fullsized': '2024-09-17T03:15:37.340186+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T03:15:43.078 INFO:tasks.ceph.ceph_manager.ceph:PG 8.0 is not active+clean 2024-09-17T03:15:43.078 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.078 INFO:tasks.ceph.ceph_manager.ceph:PG 7.f is not active+clean 2024-09-17T03:15:43.078 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.340193+0000', 'last_change': '2024-09-17T03:15:37.340193+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.340193+0000', 'last_undegraded': '2024-09-17T03:15:37.340193+0000', 'last_fullsized': '2024-09-17T03:15:37.340193+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T03:15:43.078 INFO:tasks.ceph.ceph_manager.ceph:PG 8.1 is not active+clean 2024-09-17T03:15:43.078 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.078 INFO:tasks.ceph.ceph_manager.ceph:PG 7.e is not active+clean 2024-09-17T03:15:43.079 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.340448+0000', 'last_change': '2024-09-17T03:15:37.340448+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.340448+0000', 'last_undegraded': '2024-09-17T03:15:37.340448+0000', 'last_fullsized': '2024-09-17T03:15:37.340448+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-17T03:15:43.079 INFO:tasks.ceph.ceph_manager.ceph:PG 8.2 is not active+clean 2024-09-17T03:15:43.079 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.079 INFO:tasks.ceph.ceph_manager.ceph:PG 7.d is not active+clean 2024-09-17T03:15:43.079 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.348521+0000', 'last_change': '2024-09-17T03:15:37.348521+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.348521+0000', 'last_undegraded': '2024-09-17T03:15:37.348521+0000', 'last_fullsized': '2024-09-17T03:15:37.348521+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T03:15:43.079 INFO:tasks.ceph.ceph_manager.ceph:PG 8.3 is not active+clean 2024-09-17T03:15:43.079 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.079 INFO:tasks.ceph.ceph_manager.ceph:PG 7.c is not active+clean 2024-09-17T03:15:43.079 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.342484+0000', 'last_change': '2024-09-17T03:15:37.342484+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.342484+0000', 'last_undegraded': '2024-09-17T03:15:37.342484+0000', 'last_fullsized': '2024-09-17T03:15:37.342484+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:15:43.079 INFO:tasks.ceph.ceph_manager.ceph:PG 8.4 is not active+clean 2024-09-17T03:15:43.079 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.080 INFO:tasks.ceph.ceph_manager.ceph:PG 7.b is not active+clean 2024-09-17T03:15:43.080 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.345700+0000', 'last_change': '2024-09-17T03:15:37.345700+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.345700+0000', 'last_undegraded': '2024-09-17T03:15:37.345700+0000', 'last_fullsized': '2024-09-17T03:15:37.345700+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-17T03:15:43.080 INFO:tasks.ceph.ceph_manager.ceph:PG 8.5 is not active+clean 2024-09-17T03:15:43.080 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.080 INFO:tasks.ceph.ceph_manager.ceph:PG 8.6 is not active+clean 2024-09-17T03:15:43.080 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.080 INFO:tasks.ceph.ceph_manager.ceph:PG 7.9 is not active+clean 2024-09-17T03:15:43.080 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.341965+0000', 'last_change': '2024-09-17T03:15:37.341965+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.341965+0000', 'last_undegraded': '2024-09-17T03:15:37.341965+0000', 'last_fullsized': '2024-09-17T03:15:37.341965+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:15:43.080 INFO:tasks.ceph.ceph_manager.ceph:PG 8.7 is not active+clean 2024-09-17T03:15:43.080 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.081 INFO:tasks.ceph.ceph_manager.ceph:PG 7.8 is not active+clean 2024-09-17T03:15:43.081 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.342669+0000', 'last_change': '2024-09-17T03:15:37.342669+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.342669+0000', 'last_undegraded': '2024-09-17T03:15:37.342669+0000', 'last_fullsized': '2024-09-17T03:15:37.342669+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:15:43.081 INFO:tasks.ceph.ceph_manager.ceph:PG 8.8 is not active+clean 2024-09-17T03:15:43.081 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.082 INFO:tasks.ceph.ceph_manager.ceph:PG 7.7 is not active+clean 2024-09-17T03:15:43.082 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.347427+0000', 'last_change': '2024-09-17T03:15:37.347427+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.347427+0000', 'last_undegraded': '2024-09-17T03:15:37.347427+0000', 'last_fullsized': '2024-09-17T03:15:37.347427+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T03:15:43.082 INFO:tasks.ceph.ceph_manager.ceph:PG 8.a is not active+clean 2024-09-17T03:15:43.082 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.082 INFO:tasks.ceph.ceph_manager.ceph:PG 7.5 is not active+clean 2024-09-17T03:15:43.082 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.347233+0000', 'last_change': '2024-09-17T03:15:37.347233+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.347233+0000', 'last_undegraded': '2024-09-17T03:15:37.347233+0000', 'last_fullsized': '2024-09-17T03:15:37.347233+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-17T03:15:43.082 INFO:tasks.ceph.ceph_manager.ceph:PG 8.c is not active+clean 2024-09-17T03:15:43.082 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.082 INFO:tasks.ceph.ceph_manager.ceph:PG 7.3 is not active+clean 2024-09-17T03:15:43.082 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.341949+0000', 'last_change': '2024-09-17T03:15:37.341949+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.341949+0000', 'last_undegraded': '2024-09-17T03:15:37.341949+0000', 'last_fullsized': '2024-09-17T03:15:37.341949+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:15:43.082 INFO:tasks.ceph.ceph_manager.ceph:PG 8.d is not active+clean 2024-09-17T03:15:43.083 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.083 INFO:tasks.ceph.ceph_manager.ceph:PG 8.e is not active+clean 2024-09-17T03:15:43.083 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.083 INFO:tasks.ceph.ceph_manager.ceph:PG 7.1 is not active+clean 2024-09-17T03:15:43.083 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.342475+0000', 'last_change': '2024-09-17T03:15:37.342475+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.342475+0000', 'last_undegraded': '2024-09-17T03:15:37.342475+0000', 'last_fullsized': '2024-09-17T03:15:37.342475+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-17T03:15:43.083 INFO:tasks.ceph.ceph_manager.ceph:PG 8.f is not active+clean 2024-09-17T03:15:43.083 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.084 INFO:tasks.ceph.ceph_manager.ceph:PG 7.0 is not active+clean 2024-09-17T03:15:43.084 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.352546+0000', 'last_change': '2024-09-17T03:15:37.352546+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.352546+0000', 'last_undegraded': '2024-09-17T03:15:37.352546+0000', 'last_fullsized': '2024-09-17T03:15:37.352546+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-17T03:15:43.084 INFO:tasks.ceph.ceph_manager.ceph:PG 8.9 is not active+clean 2024-09-17T03:15:43.084 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-17T03:15:38.335805+0000', 'last_change': '2024-09-17T03:15:38.335805+0000', 'last_active': '2024-09-17T03:15:38.335805+0000', 'last_peered': '2024-09-17T03:15:38.335805+0000', 'last_clean': '2024-09-17T03:15:38.335805+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:38.335805+0000', 'last_undegraded': '2024-09-17T03:15:38.335805+0000', 'last_fullsized': '2024-09-17T03:15:38.335805+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:38.335805+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-17T03:15:43.084 INFO:tasks.ceph.ceph_manager.ceph:PG 7.6 is not active+clean 2024-09-17T03:15:43.084 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 47, 'state': 'creating+peering', 'last_fresh': '2024-09-17T03:15:37.343143+0000', 'last_change': '2024-09-17T03:15:37.343143+0000', 'last_active': '2024-09-17T03:15:37.333291+0000', 'last_peered': '2024-09-17T03:15:37.333291+0000', 'last_clean': '2024-09-17T03:15:37.333291+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-17T03:15:37.343143+0000', 'last_undegraded': '2024-09-17T03:15:37.343143+0000', 'last_fullsized': '2024-09-17T03:15:37.343143+0000', 'mapping_epoch': 47, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 47, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'last_clean_scrub_stamp': '2024-09-17T03:15:37.333291+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-17T03:15:43.085 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-17T03:15:43.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.216+0000 7f63345c6640 1 -- 172.21.15.87:0/607464538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f632c0d15d0 msgr2=0x7f632c0d2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:43.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.216+0000 7f63345c6640 1 --2- 172.21.15.87:0/607464538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f632c0d15d0 0x7f632c0d2320 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f631c009830 tx=0x7f631c02a480 comp rx=0 tx=0).stop 2024-09-17T03:15:43.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.216+0000 7f63345c6640 1 -- 172.21.15.87:0/607464538 shutdown_connections 2024-09-17T03:15:43.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.216+0000 7f63345c6640 1 --2- 172.21.15.87:0/607464538 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f632c161800 0x7f632c0d2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:43.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.216+0000 7f63345c6640 1 --2- 172.21.15.87:0/607464538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f632c0d15d0 0x7f632c0d2320 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:43.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.216+0000 7f63345c6640 1 -- 172.21.15.87:0/607464538 >> 172.21.15.87:0/607464538 conn(0x7f632c09d700 msgr2=0x7f632c0d44b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:43.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.216+0000 7f63345c6640 1 -- 172.21.15.87:0/607464538 shutdown_connections 2024-09-17T03:15:43.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.216+0000 7f63345c6640 1 -- 172.21.15.87:0/607464538 wait complete. 2024-09-17T03:15:43.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.218+0000 7f63345c6640 1 Processor -- start 2024-09-17T03:15:43.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.218+0000 7f63345c6640 1 -- start start 2024-09-17T03:15:43.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.219+0000 7f63345c6640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f632c0b33d0 0x7f632c150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:43.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.219+0000 7f63345c6640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f632c161800 0x7f632c0b7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:43.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.219+0000 7f63345c6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f632c1e72b0 0x7f632c1e76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:43.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.219+0000 7f63345c6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f632c0d3130 con 0x7f632c1e72b0 2024-09-17T03:15:43.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.219+0000 7f63345c6640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f632c0d2fb0 con 0x7f632c161800 2024-09-17T03:15:43.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.219+0000 7f63345c6640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f632c0d32b0 con 0x7f632c0b33d0 2024-09-17T03:15:43.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.219+0000 7f6332b63640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f632c1e72b0 0x7f632c1e76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:43.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.219+0000 7f6332b63640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f632c1e72b0 0x7f632c1e76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36160/0 (socket says 172.21.15.87:36160) 2024-09-17T03:15:43.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.219+0000 7f6332b63640 1 -- 172.21.15.87:0/289218410 learned_addr learned my addr 172.21.15.87:0/289218410 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:43.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.219+0000 7f6332362640 1 --2- 172.21.15.87:0/289218410 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f632c0b33d0 0x7f632c150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:43.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.219+0000 7f6331b61640 1 --2- 172.21.15.87:0/289218410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f632c161800 0x7f632c0b7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:43.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.219+0000 7f6332b63640 1 -- 172.21.15.87:0/289218410 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f632c0b33d0 msgr2=0x7f632c150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:43.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.219+0000 7f6332b63640 1 --2- 172.21.15.87:0/289218410 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f632c0b33d0 0x7f632c150530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:43.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.219+0000 7f6332b63640 1 -- 172.21.15.87:0/289218410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f632c161800 msgr2=0x7f632c0b7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:43.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.219+0000 7f6332b63640 1 --2- 172.21.15.87:0/289218410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f632c161800 0x7f632c0b7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:43.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.219+0000 7f6332b63640 1 -- 172.21.15.87:0/289218410 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f632c1e7c00 con 0x7f632c1e72b0 2024-09-17T03:15:43.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.220+0000 7f6332b63640 1 --2- 172.21.15.87:0/289218410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f632c1e72b0 0x7f632c1e76c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f632400eab0 tx=0x7f632400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:43.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.220+0000 7f6322ffd640 1 -- 172.21.15.87:0/289218410 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632400cea0 con 0x7f632c1e72b0 2024-09-17T03:15:43.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.220+0000 7f6322ffd640 1 -- 172.21.15.87:0/289218410 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6324004b20 con 0x7f632c1e72b0 2024-09-17T03:15:43.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.220+0000 7f6322ffd640 1 -- 172.21.15.87:0/289218410 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324005020 con 0x7f632c1e72b0 2024-09-17T03:15:43.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.220+0000 7f63345c6640 1 -- 172.21.15.87:0/289218410 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f632c0b8720 con 0x7f632c1e72b0 2024-09-17T03:15:43.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.220+0000 7f63345c6640 1 -- 172.21.15.87:0/289218410 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f632c1e9450 con 0x7f632c1e72b0 2024-09-17T03:15:43.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.221+0000 7f63345c6640 1 -- 172.21.15.87:0/289218410 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f632c0b8c40 con 0x7f632c1e72b0 2024-09-17T03:15:43.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.221+0000 7f6322ffd640 1 -- 172.21.15.87:0/289218410 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f63240051c0 con 0x7f632c1e72b0 2024-09-17T03:15:43.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.222+0000 7f6322ffd640 1 --2- 172.21.15.87:0/289218410 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6310078b90 0x7f631007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:43.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.222+0000 7f6322ffd640 1 -- 172.21.15.87:0/289218410 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f632409ba40 con 0x7f632c1e72b0 2024-09-17T03:15:43.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.222+0000 7f6332362640 1 --2- 172.21.15.87:0/289218410 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6310078b90 0x7f631007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:43.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.223+0000 7f6332362640 1 --2- 172.21.15.87:0/289218410 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6310078b90 0x7f631007b0a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f632c08cd90 tx=0x7f6328009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:43.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.226+0000 7f6322ffd640 1 -- 172.21.15.87:0/289218410 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f632c0b8c40 con 0x7f632c1e72b0 2024-09-17T03:15:43.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.470+0000 7f63345c6640 1 -- 172.21.15.87:0/289218410 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f632c0d2320 con 0x7f632c1e72b0 2024-09-17T03:15:43.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.471+0000 7f6322ffd640 1 -- 172.21.15.87:0/289218410 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1571 (secure 0 0 0) 0x7f632409be20 con 0x7f632c1e72b0 2024-09-17T03:15:43.470 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:43.470 INFO:teuthology.orchestra.run.smithi087.stdout:{"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":114,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":8,"num_up_osds":8,"osd_up_since":1726542841,"num_in_osds":8,"osd_in_since":1726542832,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":54},{"state_name":"creating+peering","count":27},{"state_name":"active+clean","count":16}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":235491328,"bytes_avail":772858621952,"bytes_total":773094113280,"unknown_pgs_ratio":0.55670100450515747,"inactive_pgs_ratio":0.27835050225257874},"fsmap":{"epoch":20,"btime":"2024-09-17T03:15:41:361821+0000","id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"b","status":"up:active","gid":5487}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-17T03:15:25.434744+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-17T03:15:43.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.473+0000 7f63345c6640 1 -- 172.21.15.87:0/289218410 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6310078b90 msgr2=0x7f631007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:43.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.473+0000 7f63345c6640 1 --2- 172.21.15.87:0/289218410 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6310078b90 0x7f631007b0a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f632c08cd90 tx=0x7f6328009040 comp rx=0 tx=0).stop 2024-09-17T03:15:43.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.473+0000 7f63345c6640 1 -- 172.21.15.87:0/289218410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f632c1e72b0 msgr2=0x7f632c1e76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:43.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.473+0000 7f63345c6640 1 --2- 172.21.15.87:0/289218410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f632c1e72b0 0x7f632c1e76c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f632400eab0 tx=0x7f632400ef70 comp rx=0 tx=0).stop 2024-09-17T03:15:43.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.473+0000 7f63345c6640 1 -- 172.21.15.87:0/289218410 shutdown_connections 2024-09-17T03:15:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.473+0000 7f63345c6640 1 --2- 172.21.15.87:0/289218410 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6310078b90 0x7f631007b0a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.473+0000 7f63345c6640 1 --2- 172.21.15.87:0/289218410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f632c1e72b0 0x7f632c1e76c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.473+0000 7f63345c6640 1 --2- 172.21.15.87:0/289218410 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f632c161800 0x7f632c0b7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.473+0000 7f63345c6640 1 --2- 172.21.15.87:0/289218410 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f632c0b33d0 0x7f632c150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.473+0000 7f63345c6640 1 -- 172.21.15.87:0/289218410 >> 172.21.15.87:0/289218410 conn(0x7f632c09d700 msgr2=0x7f632c0b23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.473+0000 7f63345c6640 1 -- 172.21.15.87:0/289218410 shutdown_connections 2024-09-17T03:15:43.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.473+0000 7f63345c6640 1 -- 172.21.15.87:0/289218410 wait complete. 2024-09-17T03:15:43.483 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-17T03:15:43.484 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:15:43.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.598+0000 7f714c2b6640 1 -- 172.21.15.87:0/3363606843 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f71440ccf80 msgr2=0x7f71440cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:43.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.598+0000 7f714c2b6640 1 --2- 172.21.15.87:0/3363606843 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f71440ccf80 0x7f71440cd390 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f713c005fd0 tx=0x7f713c02aa90 comp rx=0 tx=0).stop 2024-09-17T03:15:43.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.598+0000 7f714c2b6640 1 -- 172.21.15.87:0/3363606843 shutdown_connections 2024-09-17T03:15:43.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.598+0000 7f714c2b6640 1 --2- 172.21.15.87:0/3363606843 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f71440ccf80 0x7f71440cd390 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:43.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.598+0000 7f714c2b6640 1 --2- 172.21.15.87:0/3363606843 >> v2:172.21.15.149:3300/0 conn(0x7f71440b76d0 0x7f71440cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:43.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.598+0000 7f714c2b6640 1 -- 172.21.15.87:0/3363606843 >> 172.21.15.87:0/3363606843 conn(0x7f71440d2230 msgr2=0x7f71440d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:43.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.598+0000 7f714c2b6640 1 -- 172.21.15.87:0/3363606843 shutdown_connections 2024-09-17T03:15:43.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.598+0000 7f714c2b6640 1 -- 172.21.15.87:0/3363606843 wait complete. 2024-09-17T03:15:43.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.600+0000 7f714c2b6640 1 Processor -- start 2024-09-17T03:15:43.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.600+0000 7f714c2b6640 1 -- start start 2024-09-17T03:15:43.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.601+0000 7f714c2b6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71440b76d0 0x7f714409cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:43.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.601+0000 7f714c2b6640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71440ccf80 0x7f71441e0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:43.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.601+0000 7f714c2b6640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f71440cf340 0x7f714409e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:43.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.601+0000 7f714c2b6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f71440d1250 con 0x7f71440b76d0 2024-09-17T03:15:43.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.601+0000 7f714c2b6640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f71440b30d0 con 0x7f71440ccf80 2024-09-17T03:15:43.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.601+0000 7f714c2b6640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f71440d13d0 con 0x7f71440cf340 2024-09-17T03:15:43.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.601+0000 7f714a052640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71440b76d0 0x7f714409cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:43.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.601+0000 7f714a052640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71440b76d0 0x7f714409cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:36162/0 (socket says 172.21.15.87:36162) 2024-09-17T03:15:43.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.601+0000 7f714a052640 1 -- 172.21.15.87:0/2309871169 learned_addr learned my addr 172.21.15.87:0/2309871169 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:43.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.601+0000 7f714a853640 1 --2- 172.21.15.87:0/2309871169 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f71440cf340 0x7f714409e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:43.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.601+0000 7f7149851640 1 --2- 172.21.15.87:0/2309871169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71440ccf80 0x7f71441e0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:43.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.601+0000 7f714a052640 1 -- 172.21.15.87:0/2309871169 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f71440cf340 msgr2=0x7f714409e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:43.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.601+0000 7f714a052640 1 --2- 172.21.15.87:0/2309871169 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f71440cf340 0x7f714409e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:43.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.601+0000 7f714a052640 1 -- 172.21.15.87:0/2309871169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71440ccf80 msgr2=0x7f71441e0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:43.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.601+0000 7f714a052640 1 --2- 172.21.15.87:0/2309871169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71440ccf80 0x7f71441e0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:43.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.601+0000 7f714a052640 1 -- 172.21.15.87:0/2309871169 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714409e0c0 con 0x7f71440b76d0 2024-09-17T03:15:43.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.602+0000 7f714a052640 1 --2- 172.21.15.87:0/2309871169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71440b76d0 0x7f714409cb10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f714000ea20 tx=0x7f714000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:43.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.602+0000 7f713affd640 1 -- 172.21.15.87:0/2309871169 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714000ce40 con 0x7f71440b76d0 2024-09-17T03:15:43.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.602+0000 7f713affd640 1 -- 172.21.15.87:0/2309871169 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7140004020 con 0x7f71440b76d0 2024-09-17T03:15:43.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.602+0000 7f714c2b6640 1 -- 172.21.15.87:0/2309871169 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714409dde0 con 0x7f71440b76d0 2024-09-17T03:15:43.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.602+0000 7f713affd640 1 -- 172.21.15.87:0/2309871169 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140004470 con 0x7f71440b76d0 2024-09-17T03:15:43.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.602+0000 7f714c2b6640 1 -- 172.21.15.87:0/2309871169 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f714409c310 con 0x7f71440b76d0 2024-09-17T03:15:43.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.603+0000 7f714c2b6640 1 -- 172.21.15.87:0/2309871169 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71440cee00 con 0x7f71440b76d0 2024-09-17T03:15:43.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.603+0000 7f713affd640 1 -- 172.21.15.87:0/2309871169 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f71400035f0 con 0x7f71440b76d0 2024-09-17T03:15:43.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.604+0000 7f713affd640 1 --2- 172.21.15.87:0/2309871169 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7128078bc0 0x7f712807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:43.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.604+0000 7f713affd640 1 -- 172.21.15.87:0/2309871169 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f714009b850 con 0x7f71440b76d0 2024-09-17T03:15:43.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.604+0000 7f7149851640 1 --2- 172.21.15.87:0/2309871169 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7128078bc0 0x7f712807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:43.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.605+0000 7f7149851640 1 --2- 172.21.15.87:0/2309871169 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7128078bc0 0x7f712807b0d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f71440c5ae0 tx=0x7f7134009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:43.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.608+0000 7f713affd640 1 -- 172.21.15.87:0/2309871169 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7140063c90 con 0x7f71440b76d0 2024-09-17T03:15:43.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.781+0000 7f714c2b6640 1 -- 172.21.15.87:0/2309871169 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f71440b69f0 con 0x7f7128078bc0 2024-09-17T03:15:43.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.785+0000 7f713affd640 1 -- 172.21.15.87:0/2309871169 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251945 (secure 0 0 0) 0x7f71440b69f0 con 0x7f7128078bc0 2024-09-17T03:15:43.788 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:43.790 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-09-17T03:15:43.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.789+0000 7f714c2b6640 1 -- 172.21.15.87:0/2309871169 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7128078bc0 msgr2=0x7f712807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:43.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.789+0000 7f714c2b6640 1 --2- 172.21.15.87:0/2309871169 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7128078bc0 0x7f712807b0d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f71440c5ae0 tx=0x7f7134009040 comp rx=0 tx=0).stop 2024-09-17T03:15:43.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.789+0000 7f714c2b6640 1 -- 172.21.15.87:0/2309871169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71440b76d0 msgr2=0x7f714409cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:43.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.789+0000 7f714c2b6640 1 --2- 172.21.15.87:0/2309871169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71440b76d0 0x7f714409cb10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f714000ea20 tx=0x7f714000eee0 comp rx=0 tx=0).stop 2024-09-17T03:15:43.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.789+0000 7f714c2b6640 1 -- 172.21.15.87:0/2309871169 shutdown_connections 2024-09-17T03:15:43.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.789+0000 7f714c2b6640 1 --2- 172.21.15.87:0/2309871169 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7128078bc0 0x7f712807b0d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:43.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.789+0000 7f714c2b6640 1 --2- 172.21.15.87:0/2309871169 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f71440cf340 0x7f714409e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:43.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.789+0000 7f714c2b6640 1 --2- 172.21.15.87:0/2309871169 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71440ccf80 0x7f71441e0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:43.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.789+0000 7f714c2b6640 1 --2- 172.21.15.87:0/2309871169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71440b76d0 0x7f714409cb10 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:43.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.789+0000 7f714c2b6640 1 -- 172.21.15.87:0/2309871169 >> 172.21.15.87:0/2309871169 conn(0x7f71440d2230 msgr2=0x7f71440b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:43.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.789+0000 7f714c2b6640 1 -- 172.21.15.87:0/2309871169 shutdown_connections 2024-09-17T03:15:43.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:43.789+0000 7f714c2b6640 1 -- 172.21.15.87:0/2309871169 wait complete. 2024-09-17T03:15:43.799 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2024-09-17T03:15:43.438936+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":145,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230312,"kb_used_data":5756,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744408,"statfs":{"total":773094113280,"available":772858273792,"internally_reserved":0,"allocated":5894144,"data_stored":3095817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001552"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053951+0000","last_change":"2024-09-17T03:15:39.374148+0000","last_active":"2024-09-17T03:15:41.053951+0000","last_peered":"2024-09-17T03:15:41.053951+0000","last_clean":"2024-09-17T03:15:41.053951+0000","last_became_active":"2024-09-17T03:15:39.373627+0000","last_became_peered":"2024-09-17T03:15:39.373627+0000","last_unstale":"2024-09-17T03:15:41.053951+0000","last_undegraded":"2024-09-17T03:15:41.053951+0000","last_fullsized":"2024-09-17T03:15:41.053951+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:01.171336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052888+0000","last_change":"2024-09-17T03:15:39.366343+0000","last_active":"2024-09-17T03:15:41.052888+0000","last_peered":"2024-09-17T03:15:41.052888+0000","last_clean":"2024-09-17T03:15:41.052888+0000","last_became_active":"2024-09-17T03:15:39.365978+0000","last_became_peered":"2024-09-17T03:15:39.365978+0000","last_unstale":"2024-09-17T03:15:41.052888+0000","last_undegraded":"2024-09-17T03:15:41.052888+0000","last_fullsized":"2024-09-17T03:15:41.052888+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:46.819112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.377499+0000","last_change":"2024-09-17T03:15:39.368117+0000","last_active":"2024-09-17T03:15:39.377499+0000","last_peered":"2024-09-17T03:15:39.377499+0000","last_clean":"2024-09-17T03:15:39.377499+0000","last_became_active":"2024-09-17T03:15:39.366527+0000","last_became_peered":"2024-09-17T03:15:39.366527+0000","last_unstale":"2024-09-17T03:15:39.377499+0000","last_undegraded":"2024-09-17T03:15:39.377499+0000","last_fullsized":"2024-09-17T03:15:39.377499+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:00:48.160980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052489+0000","last_change":"2024-09-17T03:15:39.356456+0000","last_active":"2024-09-17T03:15:41.052489+0000","last_peered":"2024-09-17T03:15:41.052489+0000","last_clean":"2024-09-17T03:15:41.052489+0000","last_became_active":"2024-09-17T03:15:39.356125+0000","last_became_peered":"2024-09-17T03:15:39.356125+0000","last_unstale":"2024-09-17T03:15:41.052489+0000","last_undegraded":"2024-09-17T03:15:41.052489+0000","last_fullsized":"2024-09-17T03:15:41.052489+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:22.174542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052970+0000","last_change":"2024-09-17T03:15:39.361621+0000","last_active":"2024-09-17T03:15:41.052970+0000","last_peered":"2024-09-17T03:15:41.052970+0000","last_clean":"2024-09-17T03:15:41.052970+0000","last_became_active":"2024-09-17T03:15:39.361289+0000","last_became_peered":"2024-09-17T03:15:39.361289+0000","last_unstale":"2024-09-17T03:15:41.052970+0000","last_undegraded":"2024-09-17T03:15:41.052970+0000","last_fullsized":"2024-09-17T03:15:41.052970+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:19.018866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053116+0000","last_change":"2024-09-17T03:15:39.370906+0000","last_active":"2024-09-17T03:15:41.053116+0000","last_peered":"2024-09-17T03:15:41.053116+0000","last_clean":"2024-09-17T03:15:41.053116+0000","last_became_active":"2024-09-17T03:15:39.370571+0000","last_became_peered":"2024-09-17T03:15:39.370571+0000","last_unstale":"2024-09-17T03:15:41.053116+0000","last_undegraded":"2024-09-17T03:15:41.053116+0000","last_fullsized":"2024-09-17T03:15:41.053116+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:23:40.418527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349770+0000","last_change":"2024-09-17T03:15:39.361231+0000","last_active":"2024-09-17T03:15:40.349770+0000","last_peered":"2024-09-17T03:15:40.349770+0000","last_clean":"2024-09-17T03:15:40.349770+0000","last_became_active":"2024-09-17T03:15:39.360893+0000","last_became_peered":"2024-09-17T03:15:39.360893+0000","last_unstale":"2024-09-17T03:15:40.349770+0000","last_undegraded":"2024-09-17T03:15:40.349770+0000","last_fullsized":"2024-09-17T03:15:40.349770+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:21.793811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052780+0000","last_change":"2024-09-17T03:15:39.368970+0000","last_active":"2024-09-17T03:15:41.052780+0000","last_peered":"2024-09-17T03:15:41.052780+0000","last_clean":"2024-09-17T03:15:41.052780+0000","last_became_active":"2024-09-17T03:15:39.368640+0000","last_became_peered":"2024-09-17T03:15:39.368640+0000","last_unstale":"2024-09-17T03:15:41.052780+0000","last_undegraded":"2024-09-17T03:15:41.052780+0000","last_fullsized":"2024-09-17T03:15:41.052780+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:03:46.892831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.377520+0000","last_change":"2024-09-17T03:15:39.367252+0000","last_active":"2024-09-17T03:15:39.377520+0000","last_peered":"2024-09-17T03:15:39.377520+0000","last_clean":"2024-09-17T03:15:39.377520+0000","last_became_active":"2024-09-17T03:15:39.366982+0000","last_became_peered":"2024-09-17T03:15:39.366982+0000","last_unstale":"2024-09-17T03:15:39.377520+0000","last_undegraded":"2024-09-17T03:15:39.377520+0000","last_fullsized":"2024-09-17T03:15:39.377520+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:02:23.197700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.351241+0000","last_change":"2024-09-17T03:15:39.370362+0000","last_active":"2024-09-17T03:15:40.351241+0000","last_peered":"2024-09-17T03:15:40.351241+0000","last_clean":"2024-09-17T03:15:40.351241+0000","last_became_active":"2024-09-17T03:15:39.370066+0000","last_became_peered":"2024-09-17T03:15:39.370066+0000","last_unstale":"2024-09-17T03:15:40.351241+0000","last_undegraded":"2024-09-17T03:15:40.351241+0000","last_fullsized":"2024-09-17T03:15:40.351241+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:25:06.914610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.054162+0000","last_change":"2024-09-17T03:15:39.372253+0000","last_active":"2024-09-17T03:15:41.054162+0000","last_peered":"2024-09-17T03:15:41.054162+0000","last_clean":"2024-09-17T03:15:41.054162+0000","last_became_active":"2024-09-17T03:15:39.371928+0000","last_became_peered":"2024-09-17T03:15:39.371928+0000","last_unstale":"2024-09-17T03:15:41.054162+0000","last_undegraded":"2024-09-17T03:15:41.054162+0000","last_fullsized":"2024-09-17T03:15:41.054162+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:52:48.901834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053035+0000","last_change":"2024-09-17T03:15:39.370486+0000","last_active":"2024-09-17T03:15:41.053035+0000","last_peered":"2024-09-17T03:15:41.053035+0000","last_clean":"2024-09-17T03:15:41.053035+0000","last_became_active":"2024-09-17T03:15:39.370159+0000","last_became_peered":"2024-09-17T03:15:39.370159+0000","last_unstale":"2024-09-17T03:15:41.053035+0000","last_undegraded":"2024-09-17T03:15:41.053035+0000","last_fullsized":"2024-09-17T03:15:41.053035+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:57.415556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052994+0000","last_change":"2024-09-17T03:15:39.365668+0000","last_active":"2024-09-17T03:15:41.052994+0000","last_peered":"2024-09-17T03:15:41.052994+0000","last_clean":"2024-09-17T03:15:41.052994+0000","last_became_active":"2024-09-17T03:15:39.365333+0000","last_became_peered":"2024-09-17T03:15:39.365333+0000","last_unstale":"2024-09-17T03:15:41.052994+0000","last_undegraded":"2024-09-17T03:15:41.052994+0000","last_fullsized":"2024-09-17T03:15:41.052994+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:58:39.898750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053482+0000","last_change":"2024-09-17T03:15:39.377543+0000","last_active":"2024-09-17T03:15:41.053482+0000","last_peered":"2024-09-17T03:15:41.053482+0000","last_clean":"2024-09-17T03:15:41.053482+0000","last_became_active":"2024-09-17T03:15:39.377223+0000","last_became_peered":"2024-09-17T03:15:39.377223+0000","last_unstale":"2024-09-17T03:15:41.053482+0000","last_undegraded":"2024-09-17T03:15:41.053482+0000","last_fullsized":"2024-09-17T03:15:41.053482+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:56.482045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052012+0000","last_change":"2024-09-17T03:15:39.368516+0000","last_active":"2024-09-17T03:15:41.052012+0000","last_peered":"2024-09-17T03:15:41.052012+0000","last_clean":"2024-09-17T03:15:41.052012+0000","last_became_active":"2024-09-17T03:15:39.368215+0000","last_became_peered":"2024-09-17T03:15:39.368215+0000","last_unstale":"2024-09-17T03:15:41.052012+0000","last_undegraded":"2024-09-17T03:15:41.052012+0000","last_fullsized":"2024-09-17T03:15:41.052012+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:02:13.373957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052751+0000","last_change":"2024-09-17T03:15:39.370515+0000","last_active":"2024-09-17T03:15:41.052751+0000","last_peered":"2024-09-17T03:15:41.052751+0000","last_clean":"2024-09-17T03:15:41.052751+0000","last_became_active":"2024-09-17T03:15:39.370215+0000","last_became_peered":"2024-09-17T03:15:39.370215+0000","last_unstale":"2024-09-17T03:15:41.052751+0000","last_undegraded":"2024-09-17T03:15:41.052751+0000","last_fullsized":"2024-09-17T03:15:41.052751+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:42:12.211487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.054150+0000","last_change":"2024-09-17T03:15:39.373138+0000","last_active":"2024-09-17T03:15:41.054150+0000","last_peered":"2024-09-17T03:15:41.054150+0000","last_clean":"2024-09-17T03:15:41.054150+0000","last_became_active":"2024-09-17T03:15:39.372833+0000","last_became_peered":"2024-09-17T03:15:39.372833+0000","last_unstale":"2024-09-17T03:15:41.054150+0000","last_undegraded":"2024-09-17T03:15:41.054150+0000","last_fullsized":"2024-09-17T03:15:41.054150+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:37.912585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052246+0000","last_change":"2024-09-17T03:15:39.357944+0000","last_active":"2024-09-17T03:15:41.052246+0000","last_peered":"2024-09-17T03:15:41.052246+0000","last_clean":"2024-09-17T03:15:41.052246+0000","last_became_active":"2024-09-17T03:15:39.357633+0000","last_became_peered":"2024-09-17T03:15:39.357633+0000","last_unstale":"2024-09-17T03:15:41.052246+0000","last_undegraded":"2024-09-17T03:15:41.052246+0000","last_fullsized":"2024-09-17T03:15:41.052246+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:35:04.892705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.369870+0000","last_change":"2024-09-17T03:15:39.366135+0000","last_active":"2024-09-17T03:15:39.369870+0000","last_peered":"2024-09-17T03:15:39.369870+0000","last_clean":"2024-09-17T03:15:39.369870+0000","last_became_active":"2024-09-17T03:15:39.365800+0000","last_became_peered":"2024-09-17T03:15:39.365800+0000","last_unstale":"2024-09-17T03:15:39.369870+0000","last_undegraded":"2024-09-17T03:15:39.369870+0000","last_fullsized":"2024-09-17T03:15:39.369870+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:17:16.473419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052892+0000","last_change":"2024-09-17T03:15:39.364776+0000","last_active":"2024-09-17T03:15:41.052892+0000","last_peered":"2024-09-17T03:15:41.052892+0000","last_clean":"2024-09-17T03:15:41.052892+0000","last_became_active":"2024-09-17T03:15:39.364456+0000","last_became_peered":"2024-09-17T03:15:39.364456+0000","last_unstale":"2024-09-17T03:15:41.052892+0000","last_undegraded":"2024-09-17T03:15:41.052892+0000","last_fullsized":"2024-09-17T03:15:41.052892+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:34.994117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052236+0000","last_change":"2024-09-17T03:15:39.367850+0000","last_active":"2024-09-17T03:15:41.052236+0000","last_peered":"2024-09-17T03:15:41.052236+0000","last_clean":"2024-09-17T03:15:41.052236+0000","last_became_active":"2024-09-17T03:15:39.367392+0000","last_became_peered":"2024-09-17T03:15:39.367392+0000","last_unstale":"2024-09-17T03:15:41.052236+0000","last_undegraded":"2024-09-17T03:15:41.052236+0000","last_fullsized":"2024-09-17T03:15:41.052236+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:45:40.350494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.380167+0000","last_change":"2024-09-17T03:15:39.371434+0000","last_active":"2024-09-17T03:15:39.380167+0000","last_peered":"2024-09-17T03:15:39.380167+0000","last_clean":"2024-09-17T03:15:39.380167+0000","last_became_active":"2024-09-17T03:15:39.371098+0000","last_became_peered":"2024-09-17T03:15:39.371098+0000","last_unstale":"2024-09-17T03:15:39.380167+0000","last_undegraded":"2024-09-17T03:15:39.380167+0000","last_fullsized":"2024-09-17T03:15:39.380167+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:33.488929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.380327+0000","last_change":"2024-09-17T03:15:39.375041+0000","last_active":"2024-09-17T03:15:39.380327+0000","last_peered":"2024-09-17T03:15:39.380327+0000","last_clean":"2024-09-17T03:15:39.380327+0000","last_became_active":"2024-09-17T03:15:39.374237+0000","last_became_peered":"2024-09-17T03:15:39.374237+0000","last_unstale":"2024-09-17T03:15:39.380327+0000","last_undegraded":"2024-09-17T03:15:39.380327+0000","last_fullsized":"2024-09-17T03:15:39.380327+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:09:06.672069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.377585+0000","last_change":"2024-09-17T03:15:39.368629+0000","last_active":"2024-09-17T03:15:39.377585+0000","last_peered":"2024-09-17T03:15:39.377585+0000","last_clean":"2024-09-17T03:15:39.377585+0000","last_became_active":"2024-09-17T03:15:39.368078+0000","last_became_peered":"2024-09-17T03:15:39.368078+0000","last_unstale":"2024-09-17T03:15:39.377585+0000","last_undegraded":"2024-09-17T03:15:39.377585+0000","last_fullsized":"2024-09-17T03:15:39.377585+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:52:24.126150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.054058+0000","last_change":"2024-09-17T03:15:39.376132+0000","last_active":"2024-09-17T03:15:41.054058+0000","last_peered":"2024-09-17T03:15:41.054058+0000","last_clean":"2024-09-17T03:15:41.054058+0000","last_became_active":"2024-09-17T03:15:39.375804+0000","last_became_peered":"2024-09-17T03:15:39.375804+0000","last_unstale":"2024-09-17T03:15:41.054058+0000","last_undegraded":"2024-09-17T03:15:41.054058+0000","last_fullsized":"2024-09-17T03:15:41.054058+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:54.073848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.380304+0000","last_change":"2024-09-17T03:15:39.374436+0000","last_active":"2024-09-17T03:15:39.380304+0000","last_peered":"2024-09-17T03:15:39.380304+0000","last_clean":"2024-09-17T03:15:39.380304+0000","last_became_active":"2024-09-17T03:15:39.372074+0000","last_became_peered":"2024-09-17T03:15:39.372074+0000","last_unstale":"2024-09-17T03:15:39.380304+0000","last_undegraded":"2024-09-17T03:15:39.380304+0000","last_fullsized":"2024-09-17T03:15:39.380304+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:48:37.306791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053160+0000","last_change":"2024-09-17T03:15:39.368992+0000","last_active":"2024-09-17T03:15:41.053160+0000","last_peered":"2024-09-17T03:15:41.053160+0000","last_clean":"2024-09-17T03:15:41.053160+0000","last_became_active":"2024-09-17T03:15:39.368672+0000","last_became_peered":"2024-09-17T03:15:39.368672+0000","last_unstale":"2024-09-17T03:15:41.053160+0000","last_undegraded":"2024-09-17T03:15:41.053160+0000","last_fullsized":"2024-09-17T03:15:41.053160+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:37.122350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052894+0000","last_change":"2024-09-17T03:15:39.364008+0000","last_active":"2024-09-17T03:15:41.052894+0000","last_peered":"2024-09-17T03:15:41.052894+0000","last_clean":"2024-09-17T03:15:41.052894+0000","last_became_active":"2024-09-17T03:15:39.363552+0000","last_became_peered":"2024-09-17T03:15:39.363552+0000","last_unstale":"2024-09-17T03:15:41.052894+0000","last_undegraded":"2024-09-17T03:15:41.052894+0000","last_fullsized":"2024-09-17T03:15:41.052894+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:54.073848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053919+0000","last_change":"2024-09-17T03:15:39.368391+0000","last_active":"2024-09-17T03:15:41.053919+0000","last_peered":"2024-09-17T03:15:41.053919+0000","last_clean":"2024-09-17T03:15:41.053919+0000","last_became_active":"2024-09-17T03:15:39.368079+0000","last_became_peered":"2024-09-17T03:15:39.368079+0000","last_unstale":"2024-09-17T03:15:41.053919+0000","last_undegraded":"2024-09-17T03:15:41.053919+0000","last_fullsized":"2024-09-17T03:15:41.053919+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:54:58.869543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349072+0000","last_change":"2024-09-17T03:15:39.376277+0000","last_active":"2024-09-17T03:15:40.349072+0000","last_peered":"2024-09-17T03:15:40.349072+0000","last_clean":"2024-09-17T03:15:40.349072+0000","last_became_active":"2024-09-17T03:15:39.375967+0000","last_became_peered":"2024-09-17T03:15:39.375967+0000","last_unstale":"2024-09-17T03:15:40.349072+0000","last_undegraded":"2024-09-17T03:15:40.349072+0000","last_fullsized":"2024-09-17T03:15:40.349072+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:10:57.197397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.377548+0000","last_change":"2024-09-17T03:15:39.367577+0000","last_active":"2024-09-17T03:15:39.377548+0000","last_peered":"2024-09-17T03:15:39.377548+0000","last_clean":"2024-09-17T03:15:39.377548+0000","last_became_active":"2024-09-17T03:15:39.367310+0000","last_became_peered":"2024-09-17T03:15:39.367310+0000","last_unstale":"2024-09-17T03:15:39.377548+0000","last_undegraded":"2024-09-17T03:15:39.377548+0000","last_fullsized":"2024-09-17T03:15:39.377548+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:57:05.939726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052220+0000","last_change":"2024-09-17T03:15:39.363283+0000","last_active":"2024-09-17T03:15:41.052220+0000","last_peered":"2024-09-17T03:15:41.052220+0000","last_clean":"2024-09-17T03:15:41.052220+0000","last_became_active":"2024-09-17T03:15:39.362950+0000","last_became_peered":"2024-09-17T03:15:39.362950+0000","last_unstale":"2024-09-17T03:15:41.052220+0000","last_undegraded":"2024-09-17T03:15:41.052220+0000","last_fullsized":"2024-09-17T03:15:41.052220+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:56.482045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349265+0000","last_change":"2024-09-17T03:15:39.358893+0000","last_active":"2024-09-17T03:15:40.349265+0000","last_peered":"2024-09-17T03:15:40.349265+0000","last_clean":"2024-09-17T03:15:40.349265+0000","last_became_active":"2024-09-17T03:15:39.358610+0000","last_became_peered":"2024-09-17T03:15:39.358610+0000","last_unstale":"2024-09-17T03:15:40.349265+0000","last_undegraded":"2024-09-17T03:15:40.349265+0000","last_fullsized":"2024-09-17T03:15:40.349265+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:44:56.984403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"49'4","reported_seq":19,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.551444+0000","last_change":"2024-09-17T03:15:38.359803+0000","last_active":"2024-09-17T03:15:40.551444+0000","last_peered":"2024-09-17T03:15:40.551444+0000","last_clean":"2024-09-17T03:15:40.551444+0000","last_became_active":"2024-09-17T03:15:38.359451+0000","last_became_peered":"2024-09-17T03:15:38.359451+0000","last_unstale":"2024-09-17T03:15:40.551444+0000","last_undegraded":"2024-09-17T03:15:40.551444+0000","last_fullsized":"2024-09-17T03:15:40.551444+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:36.119836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349068+0000","last_change":"2024-09-17T03:15:39.370104+0000","last_active":"2024-09-17T03:15:40.349068+0000","last_peered":"2024-09-17T03:15:40.349068+0000","last_clean":"2024-09-17T03:15:40.349068+0000","last_became_active":"2024-09-17T03:15:39.369814+0000","last_became_peered":"2024-09-17T03:15:39.369814+0000","last_unstale":"2024-09-17T03:15:40.349068+0000","last_undegraded":"2024-09-17T03:15:40.349068+0000","last_fullsized":"2024-09-17T03:15:40.349068+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:44.003638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"49'1","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053149+0000","last_change":"2024-09-17T03:15:38.367388+0000","last_active":"2024-09-17T03:15:41.053149+0000","last_peered":"2024-09-17T03:15:41.053149+0000","last_clean":"2024-09-17T03:15:41.053149+0000","last_became_active":"2024-09-17T03:15:38.367110+0000","last_became_peered":"2024-09-17T03:15:38.367110+0000","last_unstale":"2024-09-17T03:15:41.053149+0000","last_undegraded":"2024-09-17T03:15:41.053149+0000","last_fullsized":"2024-09-17T03:15:41.053149+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:19:05.472318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349140+0000","last_change":"2024-09-17T03:15:39.365383+0000","last_active":"2024-09-17T03:15:40.349140+0000","last_peered":"2024-09-17T03:15:40.349140+0000","last_clean":"2024-09-17T03:15:40.349140+0000","last_became_active":"2024-09-17T03:15:39.365063+0000","last_became_peered":"2024-09-17T03:15:39.365063+0000","last_unstale":"2024-09-17T03:15:40.349140+0000","last_undegraded":"2024-09-17T03:15:40.349140+0000","last_fullsized":"2024-09-17T03:15:40.349140+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:05:51.138576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"49'2","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.350937+0000","last_change":"2024-09-17T03:15:38.353637+0000","last_active":"2024-09-17T03:15:40.350937+0000","last_peered":"2024-09-17T03:15:40.350937+0000","last_clean":"2024-09-17T03:15:40.350937+0000","last_became_active":"2024-09-17T03:15:38.353275+0000","last_became_peered":"2024-09-17T03:15:38.353275+0000","last_unstale":"2024-09-17T03:15:40.350937+0000","last_undegraded":"2024-09-17T03:15:40.350937+0000","last_fullsized":"2024-09-17T03:15:40.350937+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:55.669124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349618+0000","last_change":"2024-09-17T03:15:39.373367+0000","last_active":"2024-09-17T03:15:40.349618+0000","last_peered":"2024-09-17T03:15:40.349618+0000","last_clean":"2024-09-17T03:15:40.349618+0000","last_became_active":"2024-09-17T03:15:39.373054+0000","last_became_peered":"2024-09-17T03:15:39.373054+0000","last_unstale":"2024-09-17T03:15:40.349618+0000","last_undegraded":"2024-09-17T03:15:40.349618+0000","last_fullsized":"2024-09-17T03:15:40.349618+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:32.299937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"49'2","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052976+0000","last_change":"2024-09-17T03:15:38.364110+0000","last_active":"2024-09-17T03:15:41.052976+0000","last_peered":"2024-09-17T03:15:41.052976+0000","last_clean":"2024-09-17T03:15:41.052976+0000","last_became_active":"2024-09-17T03:15:38.363709+0000","last_became_peered":"2024-09-17T03:15:38.363709+0000","last_unstale":"2024-09-17T03:15:41.052976+0000","last_undegraded":"2024-09-17T03:15:41.052976+0000","last_fullsized":"2024-09-17T03:15:41.052976+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:25.282860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.551491+0000","last_change":"2024-09-17T03:15:39.365594+0000","last_active":"2024-09-17T03:15:40.551491+0000","last_peered":"2024-09-17T03:15:40.551491+0000","last_clean":"2024-09-17T03:15:40.551491+0000","last_became_active":"2024-09-17T03:15:39.365177+0000","last_became_peered":"2024-09-17T03:15:39.365177+0000","last_unstale":"2024-09-17T03:15:40.551491+0000","last_undegraded":"2024-09-17T03:15:40.551491+0000","last_fullsized":"2024-09-17T03:15:40.551491+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:05.911970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"49'1","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.384912+0000","last_change":"2024-09-17T03:15:38.358694+0000","last_active":"2024-09-17T03:15:39.384912+0000","last_peered":"2024-09-17T03:15:39.384912+0000","last_clean":"2024-09-17T03:15:39.384912+0000","last_became_active":"2024-09-17T03:15:38.358307+0000","last_became_peered":"2024-09-17T03:15:38.358307+0000","last_unstale":"2024-09-17T03:15:39.384912+0000","last_undegraded":"2024-09-17T03:15:39.384912+0000","last_fullsized":"2024-09-17T03:15:39.384912+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:15.429280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349167+0000","last_change":"2024-09-17T03:15:39.370595+0000","last_active":"2024-09-17T03:15:40.349167+0000","last_peered":"2024-09-17T03:15:40.349167+0000","last_clean":"2024-09-17T03:15:40.349167+0000","last_became_active":"2024-09-17T03:15:39.370303+0000","last_became_peered":"2024-09-17T03:15:39.370303+0000","last_unstale":"2024-09-17T03:15:40.349167+0000","last_undegraded":"2024-09-17T03:15:40.349167+0000","last_fullsized":"2024-09-17T03:15:40.349167+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:28:43.310534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"49'1","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349142+0000","last_change":"2024-09-17T03:15:38.364539+0000","last_active":"2024-09-17T03:15:40.349142+0000","last_peered":"2024-09-17T03:15:40.349142+0000","last_clean":"2024-09-17T03:15:40.349142+0000","last_became_active":"2024-09-17T03:15:38.364231+0000","last_became_peered":"2024-09-17T03:15:38.364231+0000","last_unstale":"2024-09-17T03:15:40.349142+0000","last_undegraded":"2024-09-17T03:15:40.349142+0000","last_fullsized":"2024-09-17T03:15:40.349142+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:05.046048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.351243+0000","last_change":"2024-09-17T03:15:39.373878+0000","last_active":"2024-09-17T03:15:40.351243+0000","last_peered":"2024-09-17T03:15:40.351243+0000","last_clean":"2024-09-17T03:15:40.351243+0000","last_became_active":"2024-09-17T03:15:39.373544+0000","last_became_peered":"2024-09-17T03:15:39.373544+0000","last_unstale":"2024-09-17T03:15:40.351243+0000","last_undegraded":"2024-09-17T03:15:40.351243+0000","last_fullsized":"2024-09-17T03:15:40.351243+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:29:54.548397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.348925+0000","last_change":"2024-09-17T03:15:38.360772+0000","last_active":"2024-09-17T03:15:40.348925+0000","last_peered":"2024-09-17T03:15:40.348925+0000","last_clean":"2024-09-17T03:15:40.348925+0000","last_became_active":"2024-09-17T03:15:38.360380+0000","last_became_peered":"2024-09-17T03:15:38.360380+0000","last_unstale":"2024-09-17T03:15:40.348925+0000","last_undegraded":"2024-09-17T03:15:40.348925+0000","last_fullsized":"2024-09-17T03:15:40.348925+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:09:52.649673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052445+0000","last_change":"2024-09-17T03:15:39.368825+0000","last_active":"2024-09-17T03:15:41.052445+0000","last_peered":"2024-09-17T03:15:41.052445+0000","last_clean":"2024-09-17T03:15:41.052445+0000","last_became_active":"2024-09-17T03:15:39.368515+0000","last_became_peered":"2024-09-17T03:15:39.368515+0000","last_unstale":"2024-09-17T03:15:41.052445+0000","last_undegraded":"2024-09-17T03:15:41.052445+0000","last_fullsized":"2024-09-17T03:15:41.052445+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:57.396810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.353431+0000","last_change":"2024-09-17T03:15:38.362750+0000","last_active":"2024-09-17T03:15:39.353431+0000","last_peered":"2024-09-17T03:15:39.353431+0000","last_clean":"2024-09-17T03:15:39.353431+0000","last_became_active":"2024-09-17T03:15:38.362412+0000","last_became_peered":"2024-09-17T03:15:38.362412+0000","last_unstale":"2024-09-17T03:15:39.353431+0000","last_undegraded":"2024-09-17T03:15:39.353431+0000","last_fullsized":"2024-09-17T03:15:39.353431+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:40:22.976291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.551260+0000","last_change":"2024-09-17T03:15:39.367892+0000","last_active":"2024-09-17T03:15:40.551260+0000","last_peered":"2024-09-17T03:15:40.551260+0000","last_clean":"2024-09-17T03:15:40.551260+0000","last_became_active":"2024-09-17T03:15:39.367601+0000","last_became_peered":"2024-09-17T03:15:39.367601+0000","last_unstale":"2024-09-17T03:15:40.551260+0000","last_undegraded":"2024-09-17T03:15:40.551260+0000","last_fullsized":"2024-09-17T03:15:40.551260+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:05.487903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052435+0000","last_change":"2024-09-17T03:15:38.353919+0000","last_active":"2024-09-17T03:15:41.052435+0000","last_peered":"2024-09-17T03:15:41.052435+0000","last_clean":"2024-09-17T03:15:41.052435+0000","last_became_active":"2024-09-17T03:15:38.353467+0000","last_became_peered":"2024-09-17T03:15:38.353467+0000","last_unstale":"2024-09-17T03:15:41.052435+0000","last_undegraded":"2024-09-17T03:15:41.052435+0000","last_fullsized":"2024-09-17T03:15:41.052435+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:28:36.881547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052088+0000","last_change":"2024-09-17T03:15:39.357200+0000","last_active":"2024-09-17T03:15:41.052088+0000","last_peered":"2024-09-17T03:15:41.052088+0000","last_clean":"2024-09-17T03:15:41.052088+0000","last_became_active":"2024-09-17T03:15:39.356901+0000","last_became_peered":"2024-09-17T03:15:39.356901+0000","last_unstale":"2024-09-17T03:15:41.052088+0000","last_undegraded":"2024-09-17T03:15:41.052088+0000","last_fullsized":"2024-09-17T03:15:41.052088+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:14.166383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052136+0000","last_change":"2024-09-17T03:15:38.360165+0000","last_active":"2024-09-17T03:15:41.052136+0000","last_peered":"2024-09-17T03:15:41.052136+0000","last_clean":"2024-09-17T03:15:41.052136+0000","last_became_active":"2024-09-17T03:15:38.359882+0000","last_became_peered":"2024-09-17T03:15:38.359882+0000","last_unstale":"2024-09-17T03:15:41.052136+0000","last_undegraded":"2024-09-17T03:15:41.052136+0000","last_fullsized":"2024-09-17T03:15:41.052136+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:40.901383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053830+0000","last_change":"2024-09-17T03:15:39.371776+0000","last_active":"2024-09-17T03:15:41.053830+0000","last_peered":"2024-09-17T03:15:41.053830+0000","last_clean":"2024-09-17T03:15:41.053830+0000","last_became_active":"2024-09-17T03:15:39.371463+0000","last_became_peered":"2024-09-17T03:15:39.371463+0000","last_unstale":"2024-09-17T03:15:41.053830+0000","last_undegraded":"2024-09-17T03:15:41.053830+0000","last_fullsized":"2024-09-17T03:15:41.053830+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:34:19.482479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"49'3","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349249+0000","last_change":"2024-09-17T03:15:38.358220+0000","last_active":"2024-09-17T03:15:40.349249+0000","last_peered":"2024-09-17T03:15:40.349249+0000","last_clean":"2024-09-17T03:15:40.349249+0000","last_became_active":"2024-09-17T03:15:38.357896+0000","last_became_peered":"2024-09-17T03:15:38.357896+0000","last_unstale":"2024-09-17T03:15:40.349249+0000","last_undegraded":"2024-09-17T03:15:40.349249+0000","last_fullsized":"2024-09-17T03:15:40.349249+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:20:03.931285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052862+0000","last_change":"2024-09-17T03:15:39.372484+0000","last_active":"2024-09-17T03:15:41.052862+0000","last_peered":"2024-09-17T03:15:41.052862+0000","last_clean":"2024-09-17T03:15:41.052862+0000","last_became_active":"2024-09-17T03:15:39.372190+0000","last_became_peered":"2024-09-17T03:15:39.372190+0000","last_unstale":"2024-09-17T03:15:41.052862+0000","last_undegraded":"2024-09-17T03:15:41.052862+0000","last_fullsized":"2024-09-17T03:15:41.052862+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:54:58.869543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"49'6","reported_seq":21,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.350871+0000","last_change":"2024-09-17T03:15:38.358821+0000","last_active":"2024-09-17T03:15:40.350871+0000","last_peered":"2024-09-17T03:15:40.350871+0000","last_clean":"2024-09-17T03:15:40.350871+0000","last_became_active":"2024-09-17T03:15:38.357906+0000","last_became_peered":"2024-09-17T03:15:38.357906+0000","last_unstale":"2024-09-17T03:15:40.350871+0000","last_undegraded":"2024-09-17T03:15:40.350871+0000","last_fullsized":"2024-09-17T03:15:40.350871+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:46:36.540709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349274+0000","last_change":"2024-09-17T03:15:39.367397+0000","last_active":"2024-09-17T03:15:40.349274+0000","last_peered":"2024-09-17T03:15:40.349274+0000","last_clean":"2024-09-17T03:15:40.349274+0000","last_became_active":"2024-09-17T03:15:39.367115+0000","last_became_peered":"2024-09-17T03:15:39.367115+0000","last_unstale":"2024-09-17T03:15:40.349274+0000","last_undegraded":"2024-09-17T03:15:40.349274+0000","last_fullsized":"2024-09-17T03:15:40.349274+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:24.912968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"49'2","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052950+0000","last_change":"2024-09-17T03:15:38.364782+0000","last_active":"2024-09-17T03:15:41.052950+0000","last_peered":"2024-09-17T03:15:41.052950+0000","last_clean":"2024-09-17T03:15:41.052950+0000","last_became_active":"2024-09-17T03:15:38.364489+0000","last_became_peered":"2024-09-17T03:15:38.364489+0000","last_unstale":"2024-09-17T03:15:41.052950+0000","last_undegraded":"2024-09-17T03:15:41.052950+0000","last_fullsized":"2024-09-17T03:15:41.052950+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:54.831351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.054249+0000","last_change":"2024-09-17T03:15:39.377971+0000","last_active":"2024-09-17T03:15:41.054249+0000","last_peered":"2024-09-17T03:15:41.054249+0000","last_clean":"2024-09-17T03:15:41.054249+0000","last_became_active":"2024-09-17T03:15:39.376869+0000","last_became_peered":"2024-09-17T03:15:39.376869+0000","last_unstale":"2024-09-17T03:15:41.054249+0000","last_undegraded":"2024-09-17T03:15:41.054249+0000","last_fullsized":"2024-09-17T03:15:41.054249+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:34:54.786462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.051952+0000","last_change":"2024-09-17T03:15:38.357910+0000","last_active":"2024-09-17T03:15:41.051952+0000","last_peered":"2024-09-17T03:15:41.051952+0000","last_clean":"2024-09-17T03:15:41.051952+0000","last_became_active":"2024-09-17T03:15:38.357560+0000","last_became_peered":"2024-09-17T03:15:38.357560+0000","last_unstale":"2024-09-17T03:15:41.051952+0000","last_undegraded":"2024-09-17T03:15:41.051952+0000","last_fullsized":"2024-09-17T03:15:41.051952+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:10.852165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053748+0000","last_change":"2024-09-17T03:15:39.371171+0000","last_active":"2024-09-17T03:15:41.053748+0000","last_peered":"2024-09-17T03:15:41.053748+0000","last_clean":"2024-09-17T03:15:41.053748+0000","last_became_active":"2024-09-17T03:15:39.370860+0000","last_became_peered":"2024-09-17T03:15:39.370860+0000","last_unstale":"2024-09-17T03:15:41.053748+0000","last_undegraded":"2024-09-17T03:15:41.053748+0000","last_fullsized":"2024-09-17T03:15:41.053748+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:09.625064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052383+0000","last_change":"2024-09-17T03:15:38.364600+0000","last_active":"2024-09-17T03:15:41.052383+0000","last_peered":"2024-09-17T03:15:41.052383+0000","last_clean":"2024-09-17T03:15:41.052383+0000","last_became_active":"2024-09-17T03:15:38.364289+0000","last_became_peered":"2024-09-17T03:15:38.364289+0000","last_unstale":"2024-09-17T03:15:41.052383+0000","last_undegraded":"2024-09-17T03:15:41.052383+0000","last_fullsized":"2024-09-17T03:15:41.052383+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:45:39.347979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.380294+0000","last_change":"2024-09-17T03:15:39.374972+0000","last_active":"2024-09-17T03:15:39.380294+0000","last_peered":"2024-09-17T03:15:39.380294+0000","last_clean":"2024-09-17T03:15:39.380294+0000","last_became_active":"2024-09-17T03:15:39.374418+0000","last_became_peered":"2024-09-17T03:15:39.374418+0000","last_unstale":"2024-09-17T03:15:39.380294+0000","last_undegraded":"2024-09-17T03:15:39.380294+0000","last_fullsized":"2024-09-17T03:15:39.380294+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:53:49.563161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"49'8","reported_seq":23,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053084+0000","last_change":"2024-09-17T03:15:38.367400+0000","last_active":"2024-09-17T03:15:41.053084+0000","last_peered":"2024-09-17T03:15:41.053084+0000","last_clean":"2024-09-17T03:15:41.053084+0000","last_became_active":"2024-09-17T03:15:38.367082+0000","last_became_peered":"2024-09-17T03:15:38.367082+0000","last_unstale":"2024-09-17T03:15:41.053084+0000","last_undegraded":"2024-09-17T03:15:41.053084+0000","last_fullsized":"2024-09-17T03:15:41.053084+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:46:53.825122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349767+0000","last_change":"2024-09-17T03:15:39.362669+0000","last_active":"2024-09-17T03:15:40.349767+0000","last_peered":"2024-09-17T03:15:40.349767+0000","last_clean":"2024-09-17T03:15:40.349767+0000","last_became_active":"2024-09-17T03:15:39.361882+0000","last_became_peered":"2024-09-17T03:15:39.361882+0000","last_unstale":"2024-09-17T03:15:40.349767+0000","last_undegraded":"2024-09-17T03:15:40.349767+0000","last_fullsized":"2024-09-17T03:15:40.349767+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:19.088104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"49'1","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053626+0000","last_change":"2024-09-17T03:15:38.367369+0000","last_active":"2024-09-17T03:15:41.053626+0000","last_peered":"2024-09-17T03:15:41.053626+0000","last_clean":"2024-09-17T03:15:41.053626+0000","last_became_active":"2024-09-17T03:15:38.367032+0000","last_became_peered":"2024-09-17T03:15:38.367032+0000","last_unstale":"2024-09-17T03:15:41.053626+0000","last_undegraded":"2024-09-17T03:15:41.053626+0000","last_fullsized":"2024-09-17T03:15:41.053626+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:58.725961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053448+0000","last_change":"2024-09-17T03:15:39.379340+0000","last_active":"2024-09-17T03:15:41.053448+0000","last_peered":"2024-09-17T03:15:41.053448+0000","last_clean":"2024-09-17T03:15:41.053448+0000","last_became_active":"2024-09-17T03:15:39.379054+0000","last_became_peered":"2024-09-17T03:15:39.379054+0000","last_unstale":"2024-09-17T03:15:41.053448+0000","last_undegraded":"2024-09-17T03:15:41.053448+0000","last_fullsized":"2024-09-17T03:15:41.053448+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:59:35.887322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"49'2","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053286+0000","last_change":"2024-09-17T03:15:38.364424+0000","last_active":"2024-09-17T03:15:41.053286+0000","last_peered":"2024-09-17T03:15:41.053286+0000","last_clean":"2024-09-17T03:15:41.053286+0000","last_became_active":"2024-09-17T03:15:38.364145+0000","last_became_peered":"2024-09-17T03:15:38.364145+0000","last_unstale":"2024-09-17T03:15:41.053286+0000","last_undegraded":"2024-09-17T03:15:41.053286+0000","last_fullsized":"2024-09-17T03:15:41.053286+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:44.287924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052726+0000","last_change":"2024-09-17T03:15:39.362300+0000","last_active":"2024-09-17T03:15:41.052726+0000","last_peered":"2024-09-17T03:15:41.052726+0000","last_clean":"2024-09-17T03:15:41.052726+0000","last_became_active":"2024-09-17T03:15:39.361963+0000","last_became_peered":"2024-09-17T03:15:39.361963+0000","last_unstale":"2024-09-17T03:15:41.052726+0000","last_undegraded":"2024-09-17T03:15:41.052726+0000","last_fullsized":"2024-09-17T03:15:41.052726+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:34:19.482479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"49'1","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052350+0000","last_change":"2024-09-17T03:15:38.361043+0000","last_active":"2024-09-17T03:15:41.052350+0000","last_peered":"2024-09-17T03:15:41.052350+0000","last_clean":"2024-09-17T03:15:41.052350+0000","last_became_active":"2024-09-17T03:15:38.360725+0000","last_became_peered":"2024-09-17T03:15:38.360725+0000","last_unstale":"2024-09-17T03:15:41.052350+0000","last_undegraded":"2024-09-17T03:15:41.052350+0000","last_fullsized":"2024-09-17T03:15:41.052350+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:33:51.466444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052838+0000","last_change":"2024-09-17T03:15:39.369810+0000","last_active":"2024-09-17T03:15:41.052838+0000","last_peered":"2024-09-17T03:15:41.052838+0000","last_clean":"2024-09-17T03:15:41.052838+0000","last_became_active":"2024-09-17T03:15:39.369517+0000","last_became_peered":"2024-09-17T03:15:39.369517+0000","last_unstale":"2024-09-17T03:15:41.052838+0000","last_undegraded":"2024-09-17T03:15:41.052838+0000","last_fullsized":"2024-09-17T03:15:41.052838+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:35:04.892705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053089+0000","last_change":"2024-09-17T03:15:38.362543+0000","last_active":"2024-09-17T03:15:41.053089+0000","last_peered":"2024-09-17T03:15:41.053089+0000","last_clean":"2024-09-17T03:15:41.053089+0000","last_became_active":"2024-09-17T03:15:38.362191+0000","last_became_peered":"2024-09-17T03:15:38.362191+0000","last_unstale":"2024-09-17T03:15:41.053089+0000","last_undegraded":"2024-09-17T03:15:41.053089+0000","last_fullsized":"2024-09-17T03:15:41.053089+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:56.394296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053126+0000","last_change":"2024-09-17T03:15:39.362447+0000","last_active":"2024-09-17T03:15:41.053126+0000","last_peered":"2024-09-17T03:15:41.053126+0000","last_clean":"2024-09-17T03:15:41.053126+0000","last_became_active":"2024-09-17T03:15:39.362070+0000","last_became_peered":"2024-09-17T03:15:39.362070+0000","last_unstale":"2024-09-17T03:15:41.053126+0000","last_undegraded":"2024-09-17T03:15:41.053126+0000","last_fullsized":"2024-09-17T03:15:41.053126+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:34:54.786462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"49'2","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053793+0000","last_change":"2024-09-17T03:15:38.366985+0000","last_active":"2024-09-17T03:15:41.053793+0000","last_peered":"2024-09-17T03:15:41.053793+0000","last_clean":"2024-09-17T03:15:41.053793+0000","last_became_active":"2024-09-17T03:15:38.366625+0000","last_became_peered":"2024-09-17T03:15:38.366625+0000","last_unstale":"2024-09-17T03:15:41.053793+0000","last_undegraded":"2024-09-17T03:15:41.053793+0000","last_fullsized":"2024-09-17T03:15:41.053793+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:35:03.890191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052283+0000","last_change":"2024-09-17T03:15:39.367703+0000","last_active":"2024-09-17T03:15:41.052283+0000","last_peered":"2024-09-17T03:15:41.052283+0000","last_clean":"2024-09-17T03:15:41.052283+0000","last_became_active":"2024-09-17T03:15:39.367405+0000","last_became_peered":"2024-09-17T03:15:39.367405+0000","last_unstale":"2024-09-17T03:15:41.052283+0000","last_undegraded":"2024-09-17T03:15:41.052283+0000","last_fullsized":"2024-09-17T03:15:41.052283+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:26.285375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.357849+0000","last_change":"2024-09-17T03:15:38.361994+0000","last_active":"2024-09-17T03:15:39.357849+0000","last_peered":"2024-09-17T03:15:39.357849+0000","last_clean":"2024-09-17T03:15:39.357849+0000","last_became_active":"2024-09-17T03:15:38.361661+0000","last_became_peered":"2024-09-17T03:15:38.361661+0000","last_unstale":"2024-09-17T03:15:39.357849+0000","last_undegraded":"2024-09-17T03:15:39.357849+0000","last_fullsized":"2024-09-17T03:15:39.357849+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:46:31.452976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053673+0000","last_change":"2024-09-17T03:15:39.370709+0000","last_active":"2024-09-17T03:15:41.053673+0000","last_peered":"2024-09-17T03:15:41.053673+0000","last_clean":"2024-09-17T03:15:41.053673+0000","last_became_active":"2024-09-17T03:15:39.370394+0000","last_became_peered":"2024-09-17T03:15:39.370394+0000","last_unstale":"2024-09-17T03:15:41.053673+0000","last_undegraded":"2024-09-17T03:15:41.053673+0000","last_fullsized":"2024-09-17T03:15:41.053673+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:20:57.593683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349450+0000","last_change":"2024-09-17T03:15:38.358144+0000","last_active":"2024-09-17T03:15:40.349450+0000","last_peered":"2024-09-17T03:15:40.349450+0000","last_clean":"2024-09-17T03:15:40.349450+0000","last_became_active":"2024-09-17T03:15:38.357850+0000","last_became_peered":"2024-09-17T03:15:38.357850+0000","last_unstale":"2024-09-17T03:15:40.349450+0000","last_undegraded":"2024-09-17T03:15:40.349450+0000","last_fullsized":"2024-09-17T03:15:40.349450+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:23:08.725336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053539+0000","last_change":"2024-09-17T03:15:39.368228+0000","last_active":"2024-09-17T03:15:41.053539+0000","last_peered":"2024-09-17T03:15:41.053539+0000","last_clean":"2024-09-17T03:15:41.053539+0000","last_became_active":"2024-09-17T03:15:39.367905+0000","last_became_peered":"2024-09-17T03:15:39.367905+0000","last_unstale":"2024-09-17T03:15:41.053539+0000","last_undegraded":"2024-09-17T03:15:41.053539+0000","last_fullsized":"2024-09-17T03:15:41.053539+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:55.209682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349841+0000","last_change":"2024-09-17T03:15:38.363029+0000","last_active":"2024-09-17T03:15:40.349841+0000","last_peered":"2024-09-17T03:15:40.349841+0000","last_clean":"2024-09-17T03:15:40.349841+0000","last_became_active":"2024-09-17T03:15:38.362727+0000","last_became_peered":"2024-09-17T03:15:38.362727+0000","last_unstale":"2024-09-17T03:15:40.349841+0000","last_undegraded":"2024-09-17T03:15:40.349841+0000","last_fullsized":"2024-09-17T03:15:40.349841+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:26.398845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.051968+0000","last_change":"2024-09-17T03:15:39.364120+0000","last_active":"2024-09-17T03:15:41.051968+0000","last_peered":"2024-09-17T03:15:41.051968+0000","last_clean":"2024-09-17T03:15:41.051968+0000","last_became_active":"2024-09-17T03:15:39.363144+0000","last_became_peered":"2024-09-17T03:15:39.363144+0000","last_unstale":"2024-09-17T03:15:41.051968+0000","last_undegraded":"2024-09-17T03:15:41.051968+0000","last_fullsized":"2024-09-17T03:15:41.051968+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:56.141857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"49'2","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349139+0000","last_change":"2024-09-17T03:15:38.362454+0000","last_active":"2024-09-17T03:15:40.349139+0000","last_peered":"2024-09-17T03:15:40.349139+0000","last_clean":"2024-09-17T03:15:40.349139+0000","last_became_active":"2024-09-17T03:15:38.362154+0000","last_became_peered":"2024-09-17T03:15:38.362154+0000","last_unstale":"2024-09-17T03:15:40.349139+0000","last_undegraded":"2024-09-17T03:15:40.349139+0000","last_fullsized":"2024-09-17T03:15:40.349139+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:18.935725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.350221+0000","last_change":"2024-09-17T03:15:39.372370+0000","last_active":"2024-09-17T03:15:40.350221+0000","last_peered":"2024-09-17T03:15:40.350221+0000","last_clean":"2024-09-17T03:15:40.350221+0000","last_became_active":"2024-09-17T03:15:39.372060+0000","last_became_peered":"2024-09-17T03:15:39.372060+0000","last_unstale":"2024-09-17T03:15:40.350221+0000","last_undegraded":"2024-09-17T03:15:40.350221+0000","last_fullsized":"2024-09-17T03:15:40.350221+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:37.423937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349247+0000","last_change":"2024-09-17T03:15:38.358679+0000","last_active":"2024-09-17T03:15:40.349247+0000","last_peered":"2024-09-17T03:15:40.349247+0000","last_clean":"2024-09-17T03:15:40.349247+0000","last_became_active":"2024-09-17T03:15:38.357573+0000","last_became_peered":"2024-09-17T03:15:38.357573+0000","last_unstale":"2024-09-17T03:15:40.349247+0000","last_undegraded":"2024-09-17T03:15:40.349247+0000","last_fullsized":"2024-09-17T03:15:40.349247+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:03.043802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":10,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.375247+0000","last_change":"2024-09-17T03:15:39.366303+0000","last_active":"2024-09-17T03:15:39.375247+0000","last_peered":"2024-09-17T03:15:39.375247+0000","last_clean":"2024-09-17T03:15:39.375247+0000","last_became_active":"2024-09-17T03:15:39.365953+0000","last_became_peered":"2024-09-17T03:15:39.365953+0000","last_unstale":"2024-09-17T03:15:39.375247+0000","last_undegraded":"2024-09-17T03:15:39.375247+0000","last_fullsized":"2024-09-17T03:15:39.375247+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:35:51.052810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.356414+0000","last_change":"2024-09-17T03:15:38.362902+0000","last_active":"2024-09-17T03:15:39.356414+0000","last_peered":"2024-09-17T03:15:39.356414+0000","last_clean":"2024-09-17T03:15:39.356414+0000","last_became_active":"2024-09-17T03:15:38.362598+0000","last_became_peered":"2024-09-17T03:15:38.362598+0000","last_unstale":"2024-09-17T03:15:39.356414+0000","last_undegraded":"2024-09-17T03:15:39.356414+0000","last_fullsized":"2024-09-17T03:15:39.356414+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:56.538985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.350965+0000","last_change":"2024-09-17T03:15:39.369357+0000","last_active":"2024-09-17T03:15:40.350965+0000","last_peered":"2024-09-17T03:15:40.350965+0000","last_clean":"2024-09-17T03:15:40.350965+0000","last_became_active":"2024-09-17T03:15:39.369021+0000","last_became_peered":"2024-09-17T03:15:39.369021+0000","last_unstale":"2024-09-17T03:15:40.350965+0000","last_undegraded":"2024-09-17T03:15:40.350965+0000","last_fullsized":"2024-09-17T03:15:40.350965+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:42:48.495118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.350494+0000","last_change":"2024-09-17T03:15:38.360498+0000","last_active":"2024-09-17T03:15:40.350494+0000","last_peered":"2024-09-17T03:15:40.350494+0000","last_clean":"2024-09-17T03:15:40.350494+0000","last_became_active":"2024-09-17T03:15:38.360146+0000","last_became_peered":"2024-09-17T03:15:38.360146+0000","last_unstale":"2024-09-17T03:15:40.350494+0000","last_undegraded":"2024-09-17T03:15:40.350494+0000","last_fullsized":"2024-09-17T03:15:40.350494+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:19:19.304110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052569+0000","last_change":"2024-09-17T03:15:39.366148+0000","last_active":"2024-09-17T03:15:41.052569+0000","last_peered":"2024-09-17T03:15:41.052569+0000","last_clean":"2024-09-17T03:15:41.052569+0000","last_became_active":"2024-09-17T03:15:39.365839+0000","last_became_peered":"2024-09-17T03:15:39.365839+0000","last_unstale":"2024-09-17T03:15:41.052569+0000","last_undegraded":"2024-09-17T03:15:41.052569+0000","last_fullsized":"2024-09-17T03:15:41.052569+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:20:57.593683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-17T03:15:39.358479+0000","last_change":"2024-09-17T03:15:38.361181+0000","last_active":"2024-09-17T03:15:39.358479+0000","last_peered":"2024-09-17T03:15:39.358479+0000","last_clean":"2024-09-17T03:15:39.358479+0000","last_became_active":"2024-09-17T03:15:38.360468+0000","last_became_peered":"2024-09-17T03:15:38.360468+0000","last_unstale":"2024-09-17T03:15:39.358479+0000","last_undegraded":"2024-09-17T03:15:39.358479+0000","last_fullsized":"2024-09-17T03:15:39.358479+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:52.635369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.351456+0000","last_change":"2024-09-17T03:15:39.372260+0000","last_active":"2024-09-17T03:15:40.351456+0000","last_peered":"2024-09-17T03:15:40.351456+0000","last_clean":"2024-09-17T03:15:40.351456+0000","last_became_active":"2024-09-17T03:15:39.371961+0000","last_became_peered":"2024-09-17T03:15:39.371961+0000","last_unstale":"2024-09-17T03:15:40.351456+0000","last_undegraded":"2024-09-17T03:15:40.351456+0000","last_fullsized":"2024-09-17T03:15:40.351456+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:19.729678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052490+0000","last_change":"2024-09-17T03:15:38.364561+0000","last_active":"2024-09-17T03:15:41.052490+0000","last_peered":"2024-09-17T03:15:41.052490+0000","last_clean":"2024-09-17T03:15:41.052490+0000","last_became_active":"2024-09-17T03:15:38.364237+0000","last_became_peered":"2024-09-17T03:15:38.364237+0000","last_unstale":"2024-09-17T03:15:41.052490+0000","last_undegraded":"2024-09-17T03:15:41.052490+0000","last_fullsized":"2024-09-17T03:15:41.052490+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:52.844929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053556+0000","last_change":"2024-09-17T03:15:39.369372+0000","last_active":"2024-09-17T03:15:41.053556+0000","last_peered":"2024-09-17T03:15:41.053556+0000","last_clean":"2024-09-17T03:15:41.053556+0000","last_became_active":"2024-09-17T03:15:39.368893+0000","last_became_peered":"2024-09-17T03:15:39.368893+0000","last_unstale":"2024-09-17T03:15:41.053556+0000","last_undegraded":"2024-09-17T03:15:41.053556+0000","last_fullsized":"2024-09-17T03:15:41.053556+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:10:50.134581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.051866+0000","last_change":"2024-09-17T03:15:38.359387+0000","last_active":"2024-09-17T03:15:41.051866+0000","last_peered":"2024-09-17T03:15:41.051866+0000","last_clean":"2024-09-17T03:15:41.051866+0000","last_became_active":"2024-09-17T03:15:38.359087+0000","last_became_peered":"2024-09-17T03:15:38.359087+0000","last_unstale":"2024-09-17T03:15:41.051866+0000","last_undegraded":"2024-09-17T03:15:41.051866+0000","last_fullsized":"2024-09-17T03:15:41.051866+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:25:28.059613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.551376+0000","last_change":"2024-09-17T03:15:39.363035+0000","last_active":"2024-09-17T03:15:40.551376+0000","last_peered":"2024-09-17T03:15:40.551376+0000","last_clean":"2024-09-17T03:15:40.551376+0000","last_became_active":"2024-09-17T03:15:39.362744+0000","last_became_peered":"2024-09-17T03:15:39.362744+0000","last_unstale":"2024-09-17T03:15:40.551376+0000","last_undegraded":"2024-09-17T03:15:40.551376+0000","last_fullsized":"2024-09-17T03:15:40.551376+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:19:24.524331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"50'1","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.351379+0000","last_change":"2024-09-17T03:15:38.355270+0000","last_active":"2024-09-17T03:15:40.351379+0000","last_peered":"2024-09-17T03:15:40.351379+0000","last_clean":"2024-09-17T03:15:40.351379+0000","last_became_active":"2024-09-17T03:15:38.354970+0000","last_became_peered":"2024-09-17T03:15:38.354970+0000","last_unstale":"2024-09-17T03:15:40.351379+0000","last_undegraded":"2024-09-17T03:15:40.351379+0000","last_fullsized":"2024-09-17T03:15:40.351379+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:57:42.968414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":159,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.051661+0000","last_change":"2024-09-17T03:14:04.862167+0000","last_active":"2024-09-17T03:15:41.051661+0000","last_peered":"2024-09-17T03:15:41.051661+0000","last_clean":"2024-09-17T03:15:41.051661+0000","last_became_active":"2024-09-17T03:14:04.861731+0000","last_became_peered":"2024-09-17T03:14:04.861731+0000","last_unstale":"2024-09-17T03:15:41.051661+0000","last_undegraded":"2024-09-17T03:15:41.051661+0000","last_fullsized":"2024-09-17T03:15:41.051661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_clean_scrub_stamp":"2024-09-17T03:14:03.847721+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:32.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509465,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":593920,"data_stored":240686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":589824,"data_stored":241212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1196032,"data_stored":832700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509465,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":589824,"data_stored":240720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":589824,"data_stored":240664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509465,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1183744,"data_stored":831068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509464,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":589824,"data_stored":241212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":561152,"data_stored":227555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:15:44.691 INFO:tasks.ceph.mds.b.smithi149.stderr:2024-09-17T03:15:44.692+0000 7f789a4c2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T03:15:46.803 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:15:46.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.931+0000 7f0521e7e640 1 -- 172.21.15.87:0/1578339536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0504003a50 msgr2=0x7f0504003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:46.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.931+0000 7f0521e7e640 1 --2- 172.21.15.87:0/1578339536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0504003a50 0x7f0504003ec0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f050c0016c0 tx=0x7f0504005830 comp rx=0 tx=0).stop 2024-09-17T03:15:46.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.931+0000 7f0521e7e640 1 -- 172.21.15.87:0/1578339536 shutdown_connections 2024-09-17T03:15:46.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.931+0000 7f0521e7e640 1 --2- 172.21.15.87:0/1578339536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0504003a50 0x7f0504003ec0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:46.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.931+0000 7f0521e7e640 1 -- 172.21.15.87:0/1578339536 >> 172.21.15.87:0/1578339536 conn(0x7f051c0d1180 msgr2=0x7f051c0d1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:46.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.931+0000 7f0521e7e640 1 -- 172.21.15.87:0/1578339536 shutdown_connections 2024-09-17T03:15:46.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.931+0000 7f0521e7e640 1 -- 172.21.15.87:0/1578339536 wait complete. 2024-09-17T03:15:46.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.933+0000 7f0521e7e640 1 Processor -- start 2024-09-17T03:15:46.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.933+0000 7f0521e7e640 1 -- start start 2024-09-17T03:15:46.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.935+0000 7f0521e7e640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0504003a50 0x7f051c0b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:46.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.935+0000 7f0521e7e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f051c140570 0x7f051c1f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:46.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.935+0000 7f0521e7e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f051c1f2620 0x7f051c152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:46.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.935+0000 7f0521e7e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f051c0b4110 con 0x7f051c1f2620 2024-09-17T03:15:46.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.935+0000 7f0521e7e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f051c15f890 con 0x7f051c140570 2024-09-17T03:15:46.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.935+0000 7f0521e7e640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f051c0b4290 con 0x7f0504003a50 2024-09-17T03:15:46.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.935+0000 7f051bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f051c1f2620 0x7f051c152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:46.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.935+0000 7f051bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f051c1f2620 0x7f051c152220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42190/0 (socket says 172.21.15.87:42190) 2024-09-17T03:15:46.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.935+0000 7f051bfff640 1 -- 172.21.15.87:0/2932653674 learned_addr learned my addr 172.21.15.87:0/2932653674 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:46.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.935+0000 7f051b7fe640 1 --2- 172.21.15.87:0/2932653674 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0504003a50 0x7f051c0b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:46.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.935+0000 7f051affd640 1 --2- 172.21.15.87:0/2932653674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f051c140570 0x7f051c1f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:46.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.935+0000 7f051bfff640 1 -- 172.21.15.87:0/2932653674 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0504003a50 msgr2=0x7f051c0b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:46.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.935+0000 7f051bfff640 1 --2- 172.21.15.87:0/2932653674 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0504003a50 0x7f051c0b23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:46.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.935+0000 7f051bfff640 1 -- 172.21.15.87:0/2932653674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f051c140570 msgr2=0x7f051c1f20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:46.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.935+0000 7f051bfff640 1 --2- 172.21.15.87:0/2932653674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f051c140570 0x7f051c1f20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:46.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.935+0000 7f051bfff640 1 -- 172.21.15.87:0/2932653674 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f051c1f2a30 con 0x7f051c1f2620 2024-09-17T03:15:46.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.935+0000 7f051bfff640 1 --2- 172.21.15.87:0/2932653674 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f051c1f2620 0x7f051c152220 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0514000f80 tx=0x7f051400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:46.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.935+0000 7f0503fff640 1 -- 172.21.15.87:0/2932653674 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051400cc20 con 0x7f051c1f2620 2024-09-17T03:15:46.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.936+0000 7f0503fff640 1 -- 172.21.15.87:0/2932653674 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0514018070 con 0x7f051c1f2620 2024-09-17T03:15:46.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.936+0000 7f0521e7e640 1 -- 172.21.15.87:0/2932653674 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f051c1f2ea0 con 0x7f051c1f2620 2024-09-17T03:15:46.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.936+0000 7f0503fff640 1 -- 172.21.15.87:0/2932653674 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05140116c0 con 0x7f051c1f2620 2024-09-17T03:15:46.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.936+0000 7f0521e7e640 1 -- 172.21.15.87:0/2932653674 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f051c13fd70 con 0x7f051c1f2620 2024-09-17T03:15:46.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.937+0000 7f0521e7e640 1 -- 172.21.15.87:0/2932653674 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f051c0b47f0 con 0x7f051c1f2620 2024-09-17T03:15:46.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.937+0000 7f0503fff640 1 -- 172.21.15.87:0/2932653674 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0514011860 con 0x7f051c1f2620 2024-09-17T03:15:46.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.938+0000 7f0503fff640 1 --2- 172.21.15.87:0/2932653674 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0508078b90 0x7f050807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:46.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.938+0000 7f0503fff640 1 -- 172.21.15.87:0/2932653674 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f051409b100 con 0x7f051c1f2620 2024-09-17T03:15:46.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.938+0000 7f051b7fe640 1 --2- 172.21.15.87:0/2932653674 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0508078b90 0x7f050807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:46.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.939+0000 7f051b7fe640 1 --2- 172.21.15.87:0/2932653674 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0508078b90 0x7f050807b0a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f050c032000 tx=0x7f050c001190 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:46.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:46.943+0000 7f0503fff640 1 -- 172.21.15.87:0/2932653674 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f05140635d0 con 0x7f051c1f2620 2024-09-17T03:15:47.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.114+0000 7f0521e7e640 1 -- 172.21.15.87:0/2932653674 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f051c13f880 con 0x7f0508078b90 2024-09-17T03:15:47.117 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.119+0000 7f0503fff640 1 -- 172.21.15.87:0/2932653674 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251945 (secure 0 0 0) 0x7f051c13f880 con 0x7f0508078b90 2024-09-17T03:15:47.120 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:47.121 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-09-17T03:15:47.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.122+0000 7f0521e7e640 1 -- 172.21.15.87:0/2932653674 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0508078b90 msgr2=0x7f050807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:47.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.122+0000 7f0521e7e640 1 --2- 172.21.15.87:0/2932653674 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0508078b90 0x7f050807b0a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f050c032000 tx=0x7f050c001190 comp rx=0 tx=0).stop 2024-09-17T03:15:47.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.122+0000 7f0521e7e640 1 -- 172.21.15.87:0/2932653674 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f051c1f2620 msgr2=0x7f051c152220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:47.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.122+0000 7f0521e7e640 1 --2- 172.21.15.87:0/2932653674 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f051c1f2620 0x7f051c152220 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0514000f80 tx=0x7f051400ed10 comp rx=0 tx=0).stop 2024-09-17T03:15:47.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.122+0000 7f0521e7e640 1 -- 172.21.15.87:0/2932653674 shutdown_connections 2024-09-17T03:15:47.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.122+0000 7f0521e7e640 1 --2- 172.21.15.87:0/2932653674 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0508078b90 0x7f050807b0a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:47.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.122+0000 7f0521e7e640 1 --2- 172.21.15.87:0/2932653674 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f051c1f2620 0x7f051c152220 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:47.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.122+0000 7f0521e7e640 1 --2- 172.21.15.87:0/2932653674 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f051c140570 0x7f051c1f20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:47.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.122+0000 7f0521e7e640 1 --2- 172.21.15.87:0/2932653674 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0504003a50 0x7f051c0b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:47.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.122+0000 7f0521e7e640 1 -- 172.21.15.87:0/2932653674 >> 172.21.15.87:0/2932653674 conn(0x7f051c0d1180 msgr2=0x7f051c0b5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:47.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.122+0000 7f0521e7e640 1 -- 172.21.15.87:0/2932653674 shutdown_connections 2024-09-17T03:15:47.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.122+0000 7f0521e7e640 1 -- 172.21.15.87:0/2932653674 wait complete. 2024-09-17T03:15:47.132 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2024-09-17T03:15:45.439616+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":153,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230356,"kb_used_data":5800,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744364,"statfs":{"total":773094113280,"available":772858228736,"internally_reserved":0,"allocated":5939200,"data_stored":3110596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.095511"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053951+0000","last_change":"2024-09-17T03:15:39.374148+0000","last_active":"2024-09-17T03:15:41.053951+0000","last_peered":"2024-09-17T03:15:41.053951+0000","last_clean":"2024-09-17T03:15:41.053951+0000","last_became_active":"2024-09-17T03:15:39.373627+0000","last_became_peered":"2024-09-17T03:15:39.373627+0000","last_unstale":"2024-09-17T03:15:41.053951+0000","last_undegraded":"2024-09-17T03:15:41.053951+0000","last_fullsized":"2024-09-17T03:15:41.053951+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:01.171336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052888+0000","last_change":"2024-09-17T03:15:39.366343+0000","last_active":"2024-09-17T03:15:41.052888+0000","last_peered":"2024-09-17T03:15:41.052888+0000","last_clean":"2024-09-17T03:15:41.052888+0000","last_became_active":"2024-09-17T03:15:39.365978+0000","last_became_peered":"2024-09-17T03:15:39.365978+0000","last_unstale":"2024-09-17T03:15:41.052888+0000","last_undegraded":"2024-09-17T03:15:41.052888+0000","last_fullsized":"2024-09-17T03:15:41.052888+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:44:46.819112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349864+0000","last_change":"2024-09-17T03:15:39.368117+0000","last_active":"2024-09-17T03:15:40.349864+0000","last_peered":"2024-09-17T03:15:40.349864+0000","last_clean":"2024-09-17T03:15:40.349864+0000","last_became_active":"2024-09-17T03:15:39.366527+0000","last_became_peered":"2024-09-17T03:15:39.366527+0000","last_unstale":"2024-09-17T03:15:40.349864+0000","last_undegraded":"2024-09-17T03:15:40.349864+0000","last_fullsized":"2024-09-17T03:15:40.349864+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:30:10.466737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052489+0000","last_change":"2024-09-17T03:15:39.356456+0000","last_active":"2024-09-17T03:15:41.052489+0000","last_peered":"2024-09-17T03:15:41.052489+0000","last_clean":"2024-09-17T03:15:41.052489+0000","last_became_active":"2024-09-17T03:15:39.356125+0000","last_became_peered":"2024-09-17T03:15:39.356125+0000","last_unstale":"2024-09-17T03:15:41.052489+0000","last_undegraded":"2024-09-17T03:15:41.052489+0000","last_fullsized":"2024-09-17T03:15:41.052489+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:34:22.174542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052970+0000","last_change":"2024-09-17T03:15:39.361621+0000","last_active":"2024-09-17T03:15:41.052970+0000","last_peered":"2024-09-17T03:15:41.052970+0000","last_clean":"2024-09-17T03:15:41.052970+0000","last_became_active":"2024-09-17T03:15:39.361289+0000","last_became_peered":"2024-09-17T03:15:39.361289+0000","last_unstale":"2024-09-17T03:15:41.052970+0000","last_undegraded":"2024-09-17T03:15:41.052970+0000","last_fullsized":"2024-09-17T03:15:41.052970+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:09:19.018866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053116+0000","last_change":"2024-09-17T03:15:39.370906+0000","last_active":"2024-09-17T03:15:41.053116+0000","last_peered":"2024-09-17T03:15:41.053116+0000","last_clean":"2024-09-17T03:15:41.053116+0000","last_became_active":"2024-09-17T03:15:39.370571+0000","last_became_peered":"2024-09-17T03:15:39.370571+0000","last_unstale":"2024-09-17T03:15:41.053116+0000","last_undegraded":"2024-09-17T03:15:41.053116+0000","last_fullsized":"2024-09-17T03:15:41.053116+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:23:40.418527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349770+0000","last_change":"2024-09-17T03:15:39.361231+0000","last_active":"2024-09-17T03:15:40.349770+0000","last_peered":"2024-09-17T03:15:40.349770+0000","last_clean":"2024-09-17T03:15:40.349770+0000","last_became_active":"2024-09-17T03:15:39.360893+0000","last_became_peered":"2024-09-17T03:15:39.360893+0000","last_unstale":"2024-09-17T03:15:40.349770+0000","last_undegraded":"2024-09-17T03:15:40.349770+0000","last_fullsized":"2024-09-17T03:15:40.349770+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:16:21.793811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052780+0000","last_change":"2024-09-17T03:15:39.368970+0000","last_active":"2024-09-17T03:15:41.052780+0000","last_peered":"2024-09-17T03:15:41.052780+0000","last_clean":"2024-09-17T03:15:41.052780+0000","last_became_active":"2024-09-17T03:15:39.368640+0000","last_became_peered":"2024-09-17T03:15:39.368640+0000","last_unstale":"2024-09-17T03:15:41.052780+0000","last_undegraded":"2024-09-17T03:15:41.052780+0000","last_fullsized":"2024-09-17T03:15:41.052780+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:03:46.892831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.350094+0000","last_change":"2024-09-17T03:15:39.367252+0000","last_active":"2024-09-17T03:15:40.350094+0000","last_peered":"2024-09-17T03:15:40.350094+0000","last_clean":"2024-09-17T03:15:40.350094+0000","last_became_active":"2024-09-17T03:15:39.366982+0000","last_became_peered":"2024-09-17T03:15:39.366982+0000","last_unstale":"2024-09-17T03:15:40.350094+0000","last_undegraded":"2024-09-17T03:15:40.350094+0000","last_fullsized":"2024-09-17T03:15:40.350094+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:27:50.492319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.351241+0000","last_change":"2024-09-17T03:15:39.370362+0000","last_active":"2024-09-17T03:15:40.351241+0000","last_peered":"2024-09-17T03:15:40.351241+0000","last_clean":"2024-09-17T03:15:40.351241+0000","last_became_active":"2024-09-17T03:15:39.370066+0000","last_became_peered":"2024-09-17T03:15:39.370066+0000","last_unstale":"2024-09-17T03:15:40.351241+0000","last_undegraded":"2024-09-17T03:15:40.351241+0000","last_fullsized":"2024-09-17T03:15:40.351241+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:25:06.914610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.054162+0000","last_change":"2024-09-17T03:15:39.372253+0000","last_active":"2024-09-17T03:15:41.054162+0000","last_peered":"2024-09-17T03:15:41.054162+0000","last_clean":"2024-09-17T03:15:41.054162+0000","last_became_active":"2024-09-17T03:15:39.371928+0000","last_became_peered":"2024-09-17T03:15:39.371928+0000","last_unstale":"2024-09-17T03:15:41.054162+0000","last_undegraded":"2024-09-17T03:15:41.054162+0000","last_fullsized":"2024-09-17T03:15:41.054162+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:52:48.901834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053035+0000","last_change":"2024-09-17T03:15:39.370486+0000","last_active":"2024-09-17T03:15:41.053035+0000","last_peered":"2024-09-17T03:15:41.053035+0000","last_clean":"2024-09-17T03:15:41.053035+0000","last_became_active":"2024-09-17T03:15:39.370159+0000","last_became_peered":"2024-09-17T03:15:39.370159+0000","last_unstale":"2024-09-17T03:15:41.053035+0000","last_undegraded":"2024-09-17T03:15:41.053035+0000","last_fullsized":"2024-09-17T03:15:41.053035+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:17:57.415556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052994+0000","last_change":"2024-09-17T03:15:39.365668+0000","last_active":"2024-09-17T03:15:41.052994+0000","last_peered":"2024-09-17T03:15:41.052994+0000","last_clean":"2024-09-17T03:15:41.052994+0000","last_became_active":"2024-09-17T03:15:39.365333+0000","last_became_peered":"2024-09-17T03:15:39.365333+0000","last_unstale":"2024-09-17T03:15:41.052994+0000","last_undegraded":"2024-09-17T03:15:41.052994+0000","last_fullsized":"2024-09-17T03:15:41.052994+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:58:39.898750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053482+0000","last_change":"2024-09-17T03:15:39.377543+0000","last_active":"2024-09-17T03:15:41.053482+0000","last_peered":"2024-09-17T03:15:41.053482+0000","last_clean":"2024-09-17T03:15:41.053482+0000","last_became_active":"2024-09-17T03:15:39.377223+0000","last_became_peered":"2024-09-17T03:15:39.377223+0000","last_unstale":"2024-09-17T03:15:41.053482+0000","last_undegraded":"2024-09-17T03:15:41.053482+0000","last_fullsized":"2024-09-17T03:15:41.053482+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:56.482045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052012+0000","last_change":"2024-09-17T03:15:39.368516+0000","last_active":"2024-09-17T03:15:41.052012+0000","last_peered":"2024-09-17T03:15:41.052012+0000","last_clean":"2024-09-17T03:15:41.052012+0000","last_became_active":"2024-09-17T03:15:39.368215+0000","last_became_peered":"2024-09-17T03:15:39.368215+0000","last_unstale":"2024-09-17T03:15:41.052012+0000","last_undegraded":"2024-09-17T03:15:41.052012+0000","last_fullsized":"2024-09-17T03:15:41.052012+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:02:13.373957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052751+0000","last_change":"2024-09-17T03:15:39.370515+0000","last_active":"2024-09-17T03:15:41.052751+0000","last_peered":"2024-09-17T03:15:41.052751+0000","last_clean":"2024-09-17T03:15:41.052751+0000","last_became_active":"2024-09-17T03:15:39.370215+0000","last_became_peered":"2024-09-17T03:15:39.370215+0000","last_unstale":"2024-09-17T03:15:41.052751+0000","last_undegraded":"2024-09-17T03:15:41.052751+0000","last_fullsized":"2024-09-17T03:15:41.052751+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:42:12.211487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.054150+0000","last_change":"2024-09-17T03:15:39.373138+0000","last_active":"2024-09-17T03:15:41.054150+0000","last_peered":"2024-09-17T03:15:41.054150+0000","last_clean":"2024-09-17T03:15:41.054150+0000","last_became_active":"2024-09-17T03:15:39.372833+0000","last_became_peered":"2024-09-17T03:15:39.372833+0000","last_unstale":"2024-09-17T03:15:41.054150+0000","last_undegraded":"2024-09-17T03:15:41.054150+0000","last_fullsized":"2024-09-17T03:15:41.054150+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:40:37.912585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052246+0000","last_change":"2024-09-17T03:15:39.357944+0000","last_active":"2024-09-17T03:15:41.052246+0000","last_peered":"2024-09-17T03:15:41.052246+0000","last_clean":"2024-09-17T03:15:41.052246+0000","last_became_active":"2024-09-17T03:15:39.357633+0000","last_became_peered":"2024-09-17T03:15:39.357633+0000","last_unstale":"2024-09-17T03:15:41.052246+0000","last_undegraded":"2024-09-17T03:15:41.052246+0000","last_fullsized":"2024-09-17T03:15:41.052246+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:35:04.892705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.350291+0000","last_change":"2024-09-17T03:15:39.366135+0000","last_active":"2024-09-17T03:15:40.350291+0000","last_peered":"2024-09-17T03:15:40.350291+0000","last_clean":"2024-09-17T03:15:40.350291+0000","last_became_active":"2024-09-17T03:15:39.365800+0000","last_became_peered":"2024-09-17T03:15:39.365800+0000","last_unstale":"2024-09-17T03:15:40.350291+0000","last_undegraded":"2024-09-17T03:15:40.350291+0000","last_fullsized":"2024-09-17T03:15:40.350291+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:45:53.671908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052892+0000","last_change":"2024-09-17T03:15:39.364776+0000","last_active":"2024-09-17T03:15:41.052892+0000","last_peered":"2024-09-17T03:15:41.052892+0000","last_clean":"2024-09-17T03:15:41.052892+0000","last_became_active":"2024-09-17T03:15:39.364456+0000","last_became_peered":"2024-09-17T03:15:39.364456+0000","last_unstale":"2024-09-17T03:15:41.052892+0000","last_undegraded":"2024-09-17T03:15:41.052892+0000","last_fullsized":"2024-09-17T03:15:41.052892+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:19:34.994117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052236+0000","last_change":"2024-09-17T03:15:39.367850+0000","last_active":"2024-09-17T03:15:41.052236+0000","last_peered":"2024-09-17T03:15:41.052236+0000","last_clean":"2024-09-17T03:15:41.052236+0000","last_became_active":"2024-09-17T03:15:39.367392+0000","last_became_peered":"2024-09-17T03:15:39.367392+0000","last_unstale":"2024-09-17T03:15:41.052236+0000","last_undegraded":"2024-09-17T03:15:41.052236+0000","last_fullsized":"2024-09-17T03:15:41.052236+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:45:40.350494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.350740+0000","last_change":"2024-09-17T03:15:39.371434+0000","last_active":"2024-09-17T03:15:40.350740+0000","last_peered":"2024-09-17T03:15:40.350740+0000","last_clean":"2024-09-17T03:15:40.350740+0000","last_became_active":"2024-09-17T03:15:39.371098+0000","last_became_peered":"2024-09-17T03:15:39.371098+0000","last_unstale":"2024-09-17T03:15:40.350740+0000","last_undegraded":"2024-09-17T03:15:40.350740+0000","last_fullsized":"2024-09-17T03:15:40.350740+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:43:53.520858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.350112+0000","last_change":"2024-09-17T03:15:39.375041+0000","last_active":"2024-09-17T03:15:40.350112+0000","last_peered":"2024-09-17T03:15:40.350112+0000","last_clean":"2024-09-17T03:15:40.350112+0000","last_became_active":"2024-09-17T03:15:39.374237+0000","last_became_peered":"2024-09-17T03:15:39.374237+0000","last_unstale":"2024-09-17T03:15:40.350112+0000","last_undegraded":"2024-09-17T03:15:40.350112+0000","last_fullsized":"2024-09-17T03:15:40.350112+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:34:10.018635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349457+0000","last_change":"2024-09-17T03:15:39.368629+0000","last_active":"2024-09-17T03:15:40.349457+0000","last_peered":"2024-09-17T03:15:40.349457+0000","last_clean":"2024-09-17T03:15:40.349457+0000","last_became_active":"2024-09-17T03:15:39.368078+0000","last_became_peered":"2024-09-17T03:15:39.368078+0000","last_unstale":"2024-09-17T03:15:40.349457+0000","last_undegraded":"2024-09-17T03:15:40.349457+0000","last_fullsized":"2024-09-17T03:15:40.349457+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:33:52.315070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.054058+0000","last_change":"2024-09-17T03:15:39.376132+0000","last_active":"2024-09-17T03:15:41.054058+0000","last_peered":"2024-09-17T03:15:41.054058+0000","last_clean":"2024-09-17T03:15:41.054058+0000","last_became_active":"2024-09-17T03:15:39.375804+0000","last_became_peered":"2024-09-17T03:15:39.375804+0000","last_unstale":"2024-09-17T03:15:41.054058+0000","last_undegraded":"2024-09-17T03:15:41.054058+0000","last_fullsized":"2024-09-17T03:15:41.054058+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:54.073848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.350749+0000","last_change":"2024-09-17T03:15:39.374436+0000","last_active":"2024-09-17T03:15:40.350749+0000","last_peered":"2024-09-17T03:15:40.350749+0000","last_clean":"2024-09-17T03:15:40.350749+0000","last_became_active":"2024-09-17T03:15:39.372074+0000","last_became_peered":"2024-09-17T03:15:39.372074+0000","last_unstale":"2024-09-17T03:15:40.350749+0000","last_undegraded":"2024-09-17T03:15:40.350749+0000","last_fullsized":"2024-09-17T03:15:40.350749+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:31:21.610351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053160+0000","last_change":"2024-09-17T03:15:39.368992+0000","last_active":"2024-09-17T03:15:41.053160+0000","last_peered":"2024-09-17T03:15:41.053160+0000","last_clean":"2024-09-17T03:15:41.053160+0000","last_became_active":"2024-09-17T03:15:39.368672+0000","last_became_peered":"2024-09-17T03:15:39.368672+0000","last_unstale":"2024-09-17T03:15:41.053160+0000","last_undegraded":"2024-09-17T03:15:41.053160+0000","last_fullsized":"2024-09-17T03:15:41.053160+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:37.122350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052894+0000","last_change":"2024-09-17T03:15:39.364008+0000","last_active":"2024-09-17T03:15:41.052894+0000","last_peered":"2024-09-17T03:15:41.052894+0000","last_clean":"2024-09-17T03:15:41.052894+0000","last_became_active":"2024-09-17T03:15:39.363552+0000","last_became_peered":"2024-09-17T03:15:39.363552+0000","last_unstale":"2024-09-17T03:15:41.052894+0000","last_undegraded":"2024-09-17T03:15:41.052894+0000","last_fullsized":"2024-09-17T03:15:41.052894+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:54.073848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053919+0000","last_change":"2024-09-17T03:15:39.368391+0000","last_active":"2024-09-17T03:15:41.053919+0000","last_peered":"2024-09-17T03:15:41.053919+0000","last_clean":"2024-09-17T03:15:41.053919+0000","last_became_active":"2024-09-17T03:15:39.368079+0000","last_became_peered":"2024-09-17T03:15:39.368079+0000","last_unstale":"2024-09-17T03:15:41.053919+0000","last_undegraded":"2024-09-17T03:15:41.053919+0000","last_fullsized":"2024-09-17T03:15:41.053919+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:54:58.869543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349072+0000","last_change":"2024-09-17T03:15:39.376277+0000","last_active":"2024-09-17T03:15:40.349072+0000","last_peered":"2024-09-17T03:15:40.349072+0000","last_clean":"2024-09-17T03:15:40.349072+0000","last_became_active":"2024-09-17T03:15:39.375967+0000","last_became_peered":"2024-09-17T03:15:39.375967+0000","last_unstale":"2024-09-17T03:15:40.349072+0000","last_undegraded":"2024-09-17T03:15:40.349072+0000","last_fullsized":"2024-09-17T03:15:40.349072+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:10:57.197397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.350191+0000","last_change":"2024-09-17T03:15:39.367577+0000","last_active":"2024-09-17T03:15:40.350191+0000","last_peered":"2024-09-17T03:15:40.350191+0000","last_clean":"2024-09-17T03:15:40.350191+0000","last_became_active":"2024-09-17T03:15:39.367310+0000","last_became_peered":"2024-09-17T03:15:39.367310+0000","last_unstale":"2024-09-17T03:15:40.350191+0000","last_undegraded":"2024-09-17T03:15:40.350191+0000","last_fullsized":"2024-09-17T03:15:40.350191+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:27:24.180670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052220+0000","last_change":"2024-09-17T03:15:39.363283+0000","last_active":"2024-09-17T03:15:41.052220+0000","last_peered":"2024-09-17T03:15:41.052220+0000","last_clean":"2024-09-17T03:15:41.052220+0000","last_became_active":"2024-09-17T03:15:39.362950+0000","last_became_peered":"2024-09-17T03:15:39.362950+0000","last_unstale":"2024-09-17T03:15:41.052220+0000","last_undegraded":"2024-09-17T03:15:41.052220+0000","last_fullsized":"2024-09-17T03:15:41.052220+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:56.482045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349265+0000","last_change":"2024-09-17T03:15:39.358893+0000","last_active":"2024-09-17T03:15:40.349265+0000","last_peered":"2024-09-17T03:15:40.349265+0000","last_clean":"2024-09-17T03:15:40.349265+0000","last_became_active":"2024-09-17T03:15:39.358610+0000","last_became_peered":"2024-09-17T03:15:39.358610+0000","last_unstale":"2024-09-17T03:15:40.349265+0000","last_undegraded":"2024-09-17T03:15:40.349265+0000","last_fullsized":"2024-09-17T03:15:40.349265+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:44:56.984403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"49'4","reported_seq":19,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.551444+0000","last_change":"2024-09-17T03:15:38.359803+0000","last_active":"2024-09-17T03:15:40.551444+0000","last_peered":"2024-09-17T03:15:40.551444+0000","last_clean":"2024-09-17T03:15:40.551444+0000","last_became_active":"2024-09-17T03:15:38.359451+0000","last_became_peered":"2024-09-17T03:15:38.359451+0000","last_unstale":"2024-09-17T03:15:40.551444+0000","last_undegraded":"2024-09-17T03:15:40.551444+0000","last_fullsized":"2024-09-17T03:15:40.551444+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:43:36.119836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349068+0000","last_change":"2024-09-17T03:15:39.370104+0000","last_active":"2024-09-17T03:15:40.349068+0000","last_peered":"2024-09-17T03:15:40.349068+0000","last_clean":"2024-09-17T03:15:40.349068+0000","last_became_active":"2024-09-17T03:15:39.369814+0000","last_became_peered":"2024-09-17T03:15:39.369814+0000","last_unstale":"2024-09-17T03:15:40.349068+0000","last_undegraded":"2024-09-17T03:15:40.349068+0000","last_fullsized":"2024-09-17T03:15:40.349068+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:06:44.003638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"49'1","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053149+0000","last_change":"2024-09-17T03:15:38.367388+0000","last_active":"2024-09-17T03:15:41.053149+0000","last_peered":"2024-09-17T03:15:41.053149+0000","last_clean":"2024-09-17T03:15:41.053149+0000","last_became_active":"2024-09-17T03:15:38.367110+0000","last_became_peered":"2024-09-17T03:15:38.367110+0000","last_unstale":"2024-09-17T03:15:41.053149+0000","last_undegraded":"2024-09-17T03:15:41.053149+0000","last_fullsized":"2024-09-17T03:15:41.053149+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:19:05.472318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349140+0000","last_change":"2024-09-17T03:15:39.365383+0000","last_active":"2024-09-17T03:15:40.349140+0000","last_peered":"2024-09-17T03:15:40.349140+0000","last_clean":"2024-09-17T03:15:40.349140+0000","last_became_active":"2024-09-17T03:15:39.365063+0000","last_became_peered":"2024-09-17T03:15:39.365063+0000","last_unstale":"2024-09-17T03:15:40.349140+0000","last_undegraded":"2024-09-17T03:15:40.349140+0000","last_fullsized":"2024-09-17T03:15:40.349140+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:05:51.138576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"49'2","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.350937+0000","last_change":"2024-09-17T03:15:38.353637+0000","last_active":"2024-09-17T03:15:40.350937+0000","last_peered":"2024-09-17T03:15:40.350937+0000","last_clean":"2024-09-17T03:15:40.350937+0000","last_became_active":"2024-09-17T03:15:38.353275+0000","last_became_peered":"2024-09-17T03:15:38.353275+0000","last_unstale":"2024-09-17T03:15:40.350937+0000","last_undegraded":"2024-09-17T03:15:40.350937+0000","last_fullsized":"2024-09-17T03:15:40.350937+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:13:55.669124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349618+0000","last_change":"2024-09-17T03:15:39.373367+0000","last_active":"2024-09-17T03:15:40.349618+0000","last_peered":"2024-09-17T03:15:40.349618+0000","last_clean":"2024-09-17T03:15:40.349618+0000","last_became_active":"2024-09-17T03:15:39.373054+0000","last_became_peered":"2024-09-17T03:15:39.373054+0000","last_unstale":"2024-09-17T03:15:40.349618+0000","last_undegraded":"2024-09-17T03:15:40.349618+0000","last_fullsized":"2024-09-17T03:15:40.349618+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:57:32.299937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"49'2","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052976+0000","last_change":"2024-09-17T03:15:38.364110+0000","last_active":"2024-09-17T03:15:41.052976+0000","last_peered":"2024-09-17T03:15:41.052976+0000","last_clean":"2024-09-17T03:15:41.052976+0000","last_became_active":"2024-09-17T03:15:38.363709+0000","last_became_peered":"2024-09-17T03:15:38.363709+0000","last_unstale":"2024-09-17T03:15:41.052976+0000","last_undegraded":"2024-09-17T03:15:41.052976+0000","last_fullsized":"2024-09-17T03:15:41.052976+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:25.282860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.551491+0000","last_change":"2024-09-17T03:15:39.365594+0000","last_active":"2024-09-17T03:15:40.551491+0000","last_peered":"2024-09-17T03:15:40.551491+0000","last_clean":"2024-09-17T03:15:40.551491+0000","last_became_active":"2024-09-17T03:15:39.365177+0000","last_became_peered":"2024-09-17T03:15:39.365177+0000","last_unstale":"2024-09-17T03:15:40.551491+0000","last_undegraded":"2024-09-17T03:15:40.551491+0000","last_fullsized":"2024-09-17T03:15:40.551491+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:09:05.911970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"49'1","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.350325+0000","last_change":"2024-09-17T03:15:38.358694+0000","last_active":"2024-09-17T03:15:40.350325+0000","last_peered":"2024-09-17T03:15:40.350325+0000","last_clean":"2024-09-17T03:15:40.350325+0000","last_became_active":"2024-09-17T03:15:38.358307+0000","last_became_peered":"2024-09-17T03:15:38.358307+0000","last_unstale":"2024-09-17T03:15:40.350325+0000","last_undegraded":"2024-09-17T03:15:40.350325+0000","last_fullsized":"2024-09-17T03:15:40.350325+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:34:15.429280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349167+0000","last_change":"2024-09-17T03:15:39.370595+0000","last_active":"2024-09-17T03:15:40.349167+0000","last_peered":"2024-09-17T03:15:40.349167+0000","last_clean":"2024-09-17T03:15:40.349167+0000","last_became_active":"2024-09-17T03:15:39.370303+0000","last_became_peered":"2024-09-17T03:15:39.370303+0000","last_unstale":"2024-09-17T03:15:40.349167+0000","last_undegraded":"2024-09-17T03:15:40.349167+0000","last_fullsized":"2024-09-17T03:15:40.349167+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:28:43.310534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"49'1","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349142+0000","last_change":"2024-09-17T03:15:38.364539+0000","last_active":"2024-09-17T03:15:40.349142+0000","last_peered":"2024-09-17T03:15:40.349142+0000","last_clean":"2024-09-17T03:15:40.349142+0000","last_became_active":"2024-09-17T03:15:38.364231+0000","last_became_peered":"2024-09-17T03:15:38.364231+0000","last_unstale":"2024-09-17T03:15:40.349142+0000","last_undegraded":"2024-09-17T03:15:40.349142+0000","last_fullsized":"2024-09-17T03:15:40.349142+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T11:20:05.046048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.351243+0000","last_change":"2024-09-17T03:15:39.373878+0000","last_active":"2024-09-17T03:15:40.351243+0000","last_peered":"2024-09-17T03:15:40.351243+0000","last_clean":"2024-09-17T03:15:40.351243+0000","last_became_active":"2024-09-17T03:15:39.373544+0000","last_became_peered":"2024-09-17T03:15:39.373544+0000","last_unstale":"2024-09-17T03:15:40.351243+0000","last_undegraded":"2024-09-17T03:15:40.351243+0000","last_fullsized":"2024-09-17T03:15:40.351243+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:29:54.548397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.348925+0000","last_change":"2024-09-17T03:15:38.360772+0000","last_active":"2024-09-17T03:15:40.348925+0000","last_peered":"2024-09-17T03:15:40.348925+0000","last_clean":"2024-09-17T03:15:40.348925+0000","last_became_active":"2024-09-17T03:15:38.360380+0000","last_became_peered":"2024-09-17T03:15:38.360380+0000","last_unstale":"2024-09-17T03:15:40.348925+0000","last_undegraded":"2024-09-17T03:15:40.348925+0000","last_fullsized":"2024-09-17T03:15:40.348925+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:09:52.649673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052445+0000","last_change":"2024-09-17T03:15:39.368825+0000","last_active":"2024-09-17T03:15:41.052445+0000","last_peered":"2024-09-17T03:15:41.052445+0000","last_clean":"2024-09-17T03:15:41.052445+0000","last_became_active":"2024-09-17T03:15:39.368515+0000","last_became_peered":"2024-09-17T03:15:39.368515+0000","last_unstale":"2024-09-17T03:15:41.052445+0000","last_undegraded":"2024-09-17T03:15:41.052445+0000","last_fullsized":"2024-09-17T03:15:41.052445+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:57.396810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349644+0000","last_change":"2024-09-17T03:15:38.362750+0000","last_active":"2024-09-17T03:15:40.349644+0000","last_peered":"2024-09-17T03:15:40.349644+0000","last_clean":"2024-09-17T03:15:40.349644+0000","last_became_active":"2024-09-17T03:15:38.362412+0000","last_became_peered":"2024-09-17T03:15:38.362412+0000","last_unstale":"2024-09-17T03:15:40.349644+0000","last_undegraded":"2024-09-17T03:15:40.349644+0000","last_fullsized":"2024-09-17T03:15:40.349644+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:40:22.976291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.551260+0000","last_change":"2024-09-17T03:15:39.367892+0000","last_active":"2024-09-17T03:15:40.551260+0000","last_peered":"2024-09-17T03:15:40.551260+0000","last_clean":"2024-09-17T03:15:40.551260+0000","last_became_active":"2024-09-17T03:15:39.367601+0000","last_became_peered":"2024-09-17T03:15:39.367601+0000","last_unstale":"2024-09-17T03:15:40.551260+0000","last_undegraded":"2024-09-17T03:15:40.551260+0000","last_fullsized":"2024-09-17T03:15:40.551260+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:05:05.487903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052435+0000","last_change":"2024-09-17T03:15:38.353919+0000","last_active":"2024-09-17T03:15:41.052435+0000","last_peered":"2024-09-17T03:15:41.052435+0000","last_clean":"2024-09-17T03:15:41.052435+0000","last_became_active":"2024-09-17T03:15:38.353467+0000","last_became_peered":"2024-09-17T03:15:38.353467+0000","last_unstale":"2024-09-17T03:15:41.052435+0000","last_undegraded":"2024-09-17T03:15:41.052435+0000","last_fullsized":"2024-09-17T03:15:41.052435+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:28:36.881547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052088+0000","last_change":"2024-09-17T03:15:39.357200+0000","last_active":"2024-09-17T03:15:41.052088+0000","last_peered":"2024-09-17T03:15:41.052088+0000","last_clean":"2024-09-17T03:15:41.052088+0000","last_became_active":"2024-09-17T03:15:39.356901+0000","last_became_peered":"2024-09-17T03:15:39.356901+0000","last_unstale":"2024-09-17T03:15:41.052088+0000","last_undegraded":"2024-09-17T03:15:41.052088+0000","last_fullsized":"2024-09-17T03:15:41.052088+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:25:14.166383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052136+0000","last_change":"2024-09-17T03:15:38.360165+0000","last_active":"2024-09-17T03:15:41.052136+0000","last_peered":"2024-09-17T03:15:41.052136+0000","last_clean":"2024-09-17T03:15:41.052136+0000","last_became_active":"2024-09-17T03:15:38.359882+0000","last_became_peered":"2024-09-17T03:15:38.359882+0000","last_unstale":"2024-09-17T03:15:41.052136+0000","last_undegraded":"2024-09-17T03:15:41.052136+0000","last_fullsized":"2024-09-17T03:15:41.052136+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:32:40.901383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053830+0000","last_change":"2024-09-17T03:15:39.371776+0000","last_active":"2024-09-17T03:15:41.053830+0000","last_peered":"2024-09-17T03:15:41.053830+0000","last_clean":"2024-09-17T03:15:41.053830+0000","last_became_active":"2024-09-17T03:15:39.371463+0000","last_became_peered":"2024-09-17T03:15:39.371463+0000","last_unstale":"2024-09-17T03:15:41.053830+0000","last_undegraded":"2024-09-17T03:15:41.053830+0000","last_fullsized":"2024-09-17T03:15:41.053830+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:34:19.482479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"49'3","reported_seq":18,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349249+0000","last_change":"2024-09-17T03:15:38.358220+0000","last_active":"2024-09-17T03:15:40.349249+0000","last_peered":"2024-09-17T03:15:40.349249+0000","last_clean":"2024-09-17T03:15:40.349249+0000","last_became_active":"2024-09-17T03:15:38.357896+0000","last_became_peered":"2024-09-17T03:15:38.357896+0000","last_unstale":"2024-09-17T03:15:40.349249+0000","last_undegraded":"2024-09-17T03:15:40.349249+0000","last_fullsized":"2024-09-17T03:15:40.349249+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:20:03.931285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052862+0000","last_change":"2024-09-17T03:15:39.372484+0000","last_active":"2024-09-17T03:15:41.052862+0000","last_peered":"2024-09-17T03:15:41.052862+0000","last_clean":"2024-09-17T03:15:41.052862+0000","last_became_active":"2024-09-17T03:15:39.372190+0000","last_became_peered":"2024-09-17T03:15:39.372190+0000","last_unstale":"2024-09-17T03:15:41.052862+0000","last_undegraded":"2024-09-17T03:15:41.052862+0000","last_fullsized":"2024-09-17T03:15:41.052862+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:54:58.869543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"49'6","reported_seq":21,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.350871+0000","last_change":"2024-09-17T03:15:38.358821+0000","last_active":"2024-09-17T03:15:40.350871+0000","last_peered":"2024-09-17T03:15:40.350871+0000","last_clean":"2024-09-17T03:15:40.350871+0000","last_became_active":"2024-09-17T03:15:38.357906+0000","last_became_peered":"2024-09-17T03:15:38.357906+0000","last_unstale":"2024-09-17T03:15:40.350871+0000","last_undegraded":"2024-09-17T03:15:40.350871+0000","last_fullsized":"2024-09-17T03:15:40.350871+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:46:36.540709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349274+0000","last_change":"2024-09-17T03:15:39.367397+0000","last_active":"2024-09-17T03:15:40.349274+0000","last_peered":"2024-09-17T03:15:40.349274+0000","last_clean":"2024-09-17T03:15:40.349274+0000","last_became_active":"2024-09-17T03:15:39.367115+0000","last_became_peered":"2024-09-17T03:15:39.367115+0000","last_unstale":"2024-09-17T03:15:40.349274+0000","last_undegraded":"2024-09-17T03:15:40.349274+0000","last_fullsized":"2024-09-17T03:15:40.349274+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:34:24.912968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"49'2","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052950+0000","last_change":"2024-09-17T03:15:38.364782+0000","last_active":"2024-09-17T03:15:41.052950+0000","last_peered":"2024-09-17T03:15:41.052950+0000","last_clean":"2024-09-17T03:15:41.052950+0000","last_became_active":"2024-09-17T03:15:38.364489+0000","last_became_peered":"2024-09-17T03:15:38.364489+0000","last_unstale":"2024-09-17T03:15:41.052950+0000","last_undegraded":"2024-09-17T03:15:41.052950+0000","last_fullsized":"2024-09-17T03:15:41.052950+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:39:54.831351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.054249+0000","last_change":"2024-09-17T03:15:39.377971+0000","last_active":"2024-09-17T03:15:41.054249+0000","last_peered":"2024-09-17T03:15:41.054249+0000","last_clean":"2024-09-17T03:15:41.054249+0000","last_became_active":"2024-09-17T03:15:39.376869+0000","last_became_peered":"2024-09-17T03:15:39.376869+0000","last_unstale":"2024-09-17T03:15:41.054249+0000","last_undegraded":"2024-09-17T03:15:41.054249+0000","last_fullsized":"2024-09-17T03:15:41.054249+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:34:54.786462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.051952+0000","last_change":"2024-09-17T03:15:38.357910+0000","last_active":"2024-09-17T03:15:41.051952+0000","last_peered":"2024-09-17T03:15:41.051952+0000","last_clean":"2024-09-17T03:15:41.051952+0000","last_became_active":"2024-09-17T03:15:38.357560+0000","last_became_peered":"2024-09-17T03:15:38.357560+0000","last_unstale":"2024-09-17T03:15:41.051952+0000","last_undegraded":"2024-09-17T03:15:41.051952+0000","last_fullsized":"2024-09-17T03:15:41.051952+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:17:10.852165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053748+0000","last_change":"2024-09-17T03:15:39.371171+0000","last_active":"2024-09-17T03:15:41.053748+0000","last_peered":"2024-09-17T03:15:41.053748+0000","last_clean":"2024-09-17T03:15:41.053748+0000","last_became_active":"2024-09-17T03:15:39.370860+0000","last_became_peered":"2024-09-17T03:15:39.370860+0000","last_unstale":"2024-09-17T03:15:41.053748+0000","last_undegraded":"2024-09-17T03:15:41.053748+0000","last_fullsized":"2024-09-17T03:15:41.053748+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:33:09.625064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052383+0000","last_change":"2024-09-17T03:15:38.364600+0000","last_active":"2024-09-17T03:15:41.052383+0000","last_peered":"2024-09-17T03:15:41.052383+0000","last_clean":"2024-09-17T03:15:41.052383+0000","last_became_active":"2024-09-17T03:15:38.364289+0000","last_became_peered":"2024-09-17T03:15:38.364289+0000","last_unstale":"2024-09-17T03:15:41.052383+0000","last_undegraded":"2024-09-17T03:15:41.052383+0000","last_fullsized":"2024-09-17T03:15:41.052383+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:45:39.347979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.350345+0000","last_change":"2024-09-17T03:15:39.374972+0000","last_active":"2024-09-17T03:15:40.350345+0000","last_peered":"2024-09-17T03:15:40.350345+0000","last_clean":"2024-09-17T03:15:40.350345+0000","last_became_active":"2024-09-17T03:15:39.374418+0000","last_became_peered":"2024-09-17T03:15:39.374418+0000","last_unstale":"2024-09-17T03:15:40.350345+0000","last_undegraded":"2024-09-17T03:15:40.350345+0000","last_fullsized":"2024-09-17T03:15:40.350345+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:12:03.435412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"49'8","reported_seq":23,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053084+0000","last_change":"2024-09-17T03:15:38.367400+0000","last_active":"2024-09-17T03:15:41.053084+0000","last_peered":"2024-09-17T03:15:41.053084+0000","last_clean":"2024-09-17T03:15:41.053084+0000","last_became_active":"2024-09-17T03:15:38.367082+0000","last_became_peered":"2024-09-17T03:15:38.367082+0000","last_unstale":"2024-09-17T03:15:41.053084+0000","last_undegraded":"2024-09-17T03:15:41.053084+0000","last_fullsized":"2024-09-17T03:15:41.053084+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:46:53.825122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349767+0000","last_change":"2024-09-17T03:15:39.362669+0000","last_active":"2024-09-17T03:15:40.349767+0000","last_peered":"2024-09-17T03:15:40.349767+0000","last_clean":"2024-09-17T03:15:40.349767+0000","last_became_active":"2024-09-17T03:15:39.361882+0000","last_became_peered":"2024-09-17T03:15:39.361882+0000","last_unstale":"2024-09-17T03:15:40.349767+0000","last_undegraded":"2024-09-17T03:15:40.349767+0000","last_fullsized":"2024-09-17T03:15:40.349767+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:50:19.088104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"49'1","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053626+0000","last_change":"2024-09-17T03:15:38.367369+0000","last_active":"2024-09-17T03:15:41.053626+0000","last_peered":"2024-09-17T03:15:41.053626+0000","last_clean":"2024-09-17T03:15:41.053626+0000","last_became_active":"2024-09-17T03:15:38.367032+0000","last_became_peered":"2024-09-17T03:15:38.367032+0000","last_unstale":"2024-09-17T03:15:41.053626+0000","last_undegraded":"2024-09-17T03:15:41.053626+0000","last_fullsized":"2024-09-17T03:15:41.053626+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T07:45:58.725961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053448+0000","last_change":"2024-09-17T03:15:39.379340+0000","last_active":"2024-09-17T03:15:41.053448+0000","last_peered":"2024-09-17T03:15:41.053448+0000","last_clean":"2024-09-17T03:15:41.053448+0000","last_became_active":"2024-09-17T03:15:39.379054+0000","last_became_peered":"2024-09-17T03:15:39.379054+0000","last_unstale":"2024-09-17T03:15:41.053448+0000","last_undegraded":"2024-09-17T03:15:41.053448+0000","last_fullsized":"2024-09-17T03:15:41.053448+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:59:35.887322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"49'2","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053286+0000","last_change":"2024-09-17T03:15:38.364424+0000","last_active":"2024-09-17T03:15:41.053286+0000","last_peered":"2024-09-17T03:15:41.053286+0000","last_clean":"2024-09-17T03:15:41.053286+0000","last_became_active":"2024-09-17T03:15:38.364145+0000","last_became_peered":"2024-09-17T03:15:38.364145+0000","last_unstale":"2024-09-17T03:15:41.053286+0000","last_undegraded":"2024-09-17T03:15:41.053286+0000","last_fullsized":"2024-09-17T03:15:41.053286+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:12:44.287924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052726+0000","last_change":"2024-09-17T03:15:39.362300+0000","last_active":"2024-09-17T03:15:41.052726+0000","last_peered":"2024-09-17T03:15:41.052726+0000","last_clean":"2024-09-17T03:15:41.052726+0000","last_became_active":"2024-09-17T03:15:39.361963+0000","last_became_peered":"2024-09-17T03:15:39.361963+0000","last_unstale":"2024-09-17T03:15:41.052726+0000","last_undegraded":"2024-09-17T03:15:41.052726+0000","last_fullsized":"2024-09-17T03:15:41.052726+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:34:19.482479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"49'1","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052350+0000","last_change":"2024-09-17T03:15:38.361043+0000","last_active":"2024-09-17T03:15:41.052350+0000","last_peered":"2024-09-17T03:15:41.052350+0000","last_clean":"2024-09-17T03:15:41.052350+0000","last_became_active":"2024-09-17T03:15:38.360725+0000","last_became_peered":"2024-09-17T03:15:38.360725+0000","last_unstale":"2024-09-17T03:15:41.052350+0000","last_undegraded":"2024-09-17T03:15:41.052350+0000","last_fullsized":"2024-09-17T03:15:41.052350+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:33:51.466444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052838+0000","last_change":"2024-09-17T03:15:39.369810+0000","last_active":"2024-09-17T03:15:41.052838+0000","last_peered":"2024-09-17T03:15:41.052838+0000","last_clean":"2024-09-17T03:15:41.052838+0000","last_became_active":"2024-09-17T03:15:39.369517+0000","last_became_peered":"2024-09-17T03:15:39.369517+0000","last_unstale":"2024-09-17T03:15:41.052838+0000","last_undegraded":"2024-09-17T03:15:41.052838+0000","last_fullsized":"2024-09-17T03:15:41.052838+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:35:04.892705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053089+0000","last_change":"2024-09-17T03:15:38.362543+0000","last_active":"2024-09-17T03:15:41.053089+0000","last_peered":"2024-09-17T03:15:41.053089+0000","last_clean":"2024-09-17T03:15:41.053089+0000","last_became_active":"2024-09-17T03:15:38.362191+0000","last_became_peered":"2024-09-17T03:15:38.362191+0000","last_unstale":"2024-09-17T03:15:41.053089+0000","last_undegraded":"2024-09-17T03:15:41.053089+0000","last_fullsized":"2024-09-17T03:15:41.053089+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:22:56.394296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053126+0000","last_change":"2024-09-17T03:15:39.362447+0000","last_active":"2024-09-17T03:15:41.053126+0000","last_peered":"2024-09-17T03:15:41.053126+0000","last_clean":"2024-09-17T03:15:41.053126+0000","last_became_active":"2024-09-17T03:15:39.362070+0000","last_became_peered":"2024-09-17T03:15:39.362070+0000","last_unstale":"2024-09-17T03:15:41.053126+0000","last_undegraded":"2024-09-17T03:15:41.053126+0000","last_fullsized":"2024-09-17T03:15:41.053126+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:34:54.786462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"49'2","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053793+0000","last_change":"2024-09-17T03:15:38.366985+0000","last_active":"2024-09-17T03:15:41.053793+0000","last_peered":"2024-09-17T03:15:41.053793+0000","last_clean":"2024-09-17T03:15:41.053793+0000","last_became_active":"2024-09-17T03:15:38.366625+0000","last_became_peered":"2024-09-17T03:15:38.366625+0000","last_unstale":"2024-09-17T03:15:41.053793+0000","last_undegraded":"2024-09-17T03:15:41.053793+0000","last_fullsized":"2024-09-17T03:15:41.053793+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T03:35:03.890191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052283+0000","last_change":"2024-09-17T03:15:39.367703+0000","last_active":"2024-09-17T03:15:41.052283+0000","last_peered":"2024-09-17T03:15:41.052283+0000","last_clean":"2024-09-17T03:15:41.052283+0000","last_became_active":"2024-09-17T03:15:39.367405+0000","last_became_peered":"2024-09-17T03:15:39.367405+0000","last_unstale":"2024-09-17T03:15:41.052283+0000","last_undegraded":"2024-09-17T03:15:41.052283+0000","last_fullsized":"2024-09-17T03:15:41.052283+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:05:26.285375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.350855+0000","last_change":"2024-09-17T03:15:38.361994+0000","last_active":"2024-09-17T03:15:40.350855+0000","last_peered":"2024-09-17T03:15:40.350855+0000","last_clean":"2024-09-17T03:15:40.350855+0000","last_became_active":"2024-09-17T03:15:38.361661+0000","last_became_peered":"2024-09-17T03:15:38.361661+0000","last_unstale":"2024-09-17T03:15:40.350855+0000","last_undegraded":"2024-09-17T03:15:40.350855+0000","last_fullsized":"2024-09-17T03:15:40.350855+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:46:31.452976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053673+0000","last_change":"2024-09-17T03:15:39.370709+0000","last_active":"2024-09-17T03:15:41.053673+0000","last_peered":"2024-09-17T03:15:41.053673+0000","last_clean":"2024-09-17T03:15:41.053673+0000","last_became_active":"2024-09-17T03:15:39.370394+0000","last_became_peered":"2024-09-17T03:15:39.370394+0000","last_unstale":"2024-09-17T03:15:41.053673+0000","last_undegraded":"2024-09-17T03:15:41.053673+0000","last_fullsized":"2024-09-17T03:15:41.053673+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:20:57.593683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349450+0000","last_change":"2024-09-17T03:15:38.358144+0000","last_active":"2024-09-17T03:15:40.349450+0000","last_peered":"2024-09-17T03:15:40.349450+0000","last_clean":"2024-09-17T03:15:40.349450+0000","last_became_active":"2024-09-17T03:15:38.357850+0000","last_became_peered":"2024-09-17T03:15:38.357850+0000","last_unstale":"2024-09-17T03:15:40.349450+0000","last_undegraded":"2024-09-17T03:15:40.349450+0000","last_fullsized":"2024-09-17T03:15:40.349450+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:23:08.725336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053539+0000","last_change":"2024-09-17T03:15:39.368228+0000","last_active":"2024-09-17T03:15:41.053539+0000","last_peered":"2024-09-17T03:15:41.053539+0000","last_clean":"2024-09-17T03:15:41.053539+0000","last_became_active":"2024-09-17T03:15:39.367905+0000","last_became_peered":"2024-09-17T03:15:39.367905+0000","last_unstale":"2024-09-17T03:15:41.053539+0000","last_undegraded":"2024-09-17T03:15:41.053539+0000","last_fullsized":"2024-09-17T03:15:41.053539+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:09:55.209682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349841+0000","last_change":"2024-09-17T03:15:38.363029+0000","last_active":"2024-09-17T03:15:40.349841+0000","last_peered":"2024-09-17T03:15:40.349841+0000","last_clean":"2024-09-17T03:15:40.349841+0000","last_became_active":"2024-09-17T03:15:38.362727+0000","last_became_peered":"2024-09-17T03:15:38.362727+0000","last_unstale":"2024-09-17T03:15:40.349841+0000","last_undegraded":"2024-09-17T03:15:40.349841+0000","last_fullsized":"2024-09-17T03:15:40.349841+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:16:26.398845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.051968+0000","last_change":"2024-09-17T03:15:39.364120+0000","last_active":"2024-09-17T03:15:41.051968+0000","last_peered":"2024-09-17T03:15:41.051968+0000","last_clean":"2024-09-17T03:15:41.051968+0000","last_became_active":"2024-09-17T03:15:39.363144+0000","last_became_peered":"2024-09-17T03:15:39.363144+0000","last_unstale":"2024-09-17T03:15:41.051968+0000","last_undegraded":"2024-09-17T03:15:41.051968+0000","last_fullsized":"2024-09-17T03:15:41.051968+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T15:00:56.141857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"49'2","reported_seq":17,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349139+0000","last_change":"2024-09-17T03:15:38.362454+0000","last_active":"2024-09-17T03:15:40.349139+0000","last_peered":"2024-09-17T03:15:40.349139+0000","last_clean":"2024-09-17T03:15:40.349139+0000","last_became_active":"2024-09-17T03:15:38.362154+0000","last_became_peered":"2024-09-17T03:15:38.362154+0000","last_unstale":"2024-09-17T03:15:40.349139+0000","last_undegraded":"2024-09-17T03:15:40.349139+0000","last_fullsized":"2024-09-17T03:15:40.349139+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:21:18.935725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.350221+0000","last_change":"2024-09-17T03:15:39.372370+0000","last_active":"2024-09-17T03:15:40.350221+0000","last_peered":"2024-09-17T03:15:40.350221+0000","last_clean":"2024-09-17T03:15:40.350221+0000","last_became_active":"2024-09-17T03:15:39.372060+0000","last_became_peered":"2024-09-17T03:15:39.372060+0000","last_unstale":"2024-09-17T03:15:40.350221+0000","last_undegraded":"2024-09-17T03:15:40.350221+0000","last_fullsized":"2024-09-17T03:15:40.350221+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:48:37.423937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.349247+0000","last_change":"2024-09-17T03:15:38.358679+0000","last_active":"2024-09-17T03:15:40.349247+0000","last_peered":"2024-09-17T03:15:40.349247+0000","last_clean":"2024-09-17T03:15:40.349247+0000","last_became_active":"2024-09-17T03:15:38.357573+0000","last_became_peered":"2024-09-17T03:15:38.357573+0000","last_unstale":"2024-09-17T03:15:40.349247+0000","last_undegraded":"2024-09-17T03:15:40.349247+0000","last_fullsized":"2024-09-17T03:15:40.349247+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:03.043802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.350149+0000","last_change":"2024-09-17T03:15:39.366303+0000","last_active":"2024-09-17T03:15:40.350149+0000","last_peered":"2024-09-17T03:15:40.350149+0000","last_clean":"2024-09-17T03:15:40.350149+0000","last_became_active":"2024-09-17T03:15:39.365953+0000","last_became_peered":"2024-09-17T03:15:39.365953+0000","last_unstale":"2024-09-17T03:15:40.350149+0000","last_undegraded":"2024-09-17T03:15:40.350149+0000","last_fullsized":"2024-09-17T03:15:40.350149+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T09:04:39.494227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.351014+0000","last_change":"2024-09-17T03:15:38.362902+0000","last_active":"2024-09-17T03:15:40.351014+0000","last_peered":"2024-09-17T03:15:40.351014+0000","last_clean":"2024-09-17T03:15:40.351014+0000","last_became_active":"2024-09-17T03:15:38.362598+0000","last_became_peered":"2024-09-17T03:15:38.362598+0000","last_unstale":"2024-09-17T03:15:40.351014+0000","last_undegraded":"2024-09-17T03:15:40.351014+0000","last_fullsized":"2024-09-17T03:15:40.351014+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:15:56.538985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.350965+0000","last_change":"2024-09-17T03:15:39.369357+0000","last_active":"2024-09-17T03:15:40.350965+0000","last_peered":"2024-09-17T03:15:40.350965+0000","last_clean":"2024-09-17T03:15:40.350965+0000","last_became_active":"2024-09-17T03:15:39.369021+0000","last_became_peered":"2024-09-17T03:15:39.369021+0000","last_unstale":"2024-09-17T03:15:40.350965+0000","last_undegraded":"2024-09-17T03:15:40.350965+0000","last_fullsized":"2024-09-17T03:15:40.350965+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:42:48.495118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.350494+0000","last_change":"2024-09-17T03:15:38.360498+0000","last_active":"2024-09-17T03:15:40.350494+0000","last_peered":"2024-09-17T03:15:40.350494+0000","last_clean":"2024-09-17T03:15:40.350494+0000","last_became_active":"2024-09-17T03:15:38.360146+0000","last_became_peered":"2024-09-17T03:15:38.360146+0000","last_unstale":"2024-09-17T03:15:40.350494+0000","last_undegraded":"2024-09-17T03:15:40.350494+0000","last_fullsized":"2024-09-17T03:15:40.350494+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T06:19:19.304110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052569+0000","last_change":"2024-09-17T03:15:39.366148+0000","last_active":"2024-09-17T03:15:41.052569+0000","last_peered":"2024-09-17T03:15:41.052569+0000","last_clean":"2024-09-17T03:15:41.052569+0000","last_became_active":"2024-09-17T03:15:39.365839+0000","last_became_peered":"2024-09-17T03:15:39.365839+0000","last_unstale":"2024-09-17T03:15:41.052569+0000","last_undegraded":"2024-09-17T03:15:41.052569+0000","last_fullsized":"2024-09-17T03:15:41.052569+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T13:20:57.593683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.350923+0000","last_change":"2024-09-17T03:15:38.361181+0000","last_active":"2024-09-17T03:15:40.350923+0000","last_peered":"2024-09-17T03:15:40.350923+0000","last_clean":"2024-09-17T03:15:40.350923+0000","last_became_active":"2024-09-17T03:15:38.360468+0000","last_became_peered":"2024-09-17T03:15:38.360468+0000","last_unstale":"2024-09-17T03:15:40.350923+0000","last_undegraded":"2024-09-17T03:15:40.350923+0000","last_fullsized":"2024-09-17T03:15:40.350923+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T05:57:52.635369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.351456+0000","last_change":"2024-09-17T03:15:39.372260+0000","last_active":"2024-09-17T03:15:40.351456+0000","last_peered":"2024-09-17T03:15:40.351456+0000","last_clean":"2024-09-17T03:15:40.351456+0000","last_became_active":"2024-09-17T03:15:39.371961+0000","last_became_peered":"2024-09-17T03:15:39.371961+0000","last_unstale":"2024-09-17T03:15:40.351456+0000","last_undegraded":"2024-09-17T03:15:40.351456+0000","last_fullsized":"2024-09-17T03:15:40.351456+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:35:19.729678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.052490+0000","last_change":"2024-09-17T03:15:38.364561+0000","last_active":"2024-09-17T03:15:41.052490+0000","last_peered":"2024-09-17T03:15:41.052490+0000","last_clean":"2024-09-17T03:15:41.052490+0000","last_became_active":"2024-09-17T03:15:38.364237+0000","last_became_peered":"2024-09-17T03:15:38.364237+0000","last_unstale":"2024-09-17T03:15:41.052490+0000","last_undegraded":"2024-09-17T03:15:41.052490+0000","last_fullsized":"2024-09-17T03:15:41.052490+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T08:00:52.844929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.053556+0000","last_change":"2024-09-17T03:15:39.369372+0000","last_active":"2024-09-17T03:15:41.053556+0000","last_peered":"2024-09-17T03:15:41.053556+0000","last_clean":"2024-09-17T03:15:41.053556+0000","last_became_active":"2024-09-17T03:15:39.368893+0000","last_became_peered":"2024-09-17T03:15:39.368893+0000","last_unstale":"2024-09-17T03:15:41.053556+0000","last_undegraded":"2024-09-17T03:15:41.053556+0000","last_fullsized":"2024-09-17T03:15:41.053556+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:10:50.134581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":15,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.051866+0000","last_change":"2024-09-17T03:15:38.359387+0000","last_active":"2024-09-17T03:15:41.051866+0000","last_peered":"2024-09-17T03:15:41.051866+0000","last_clean":"2024-09-17T03:15:41.051866+0000","last_became_active":"2024-09-17T03:15:38.359087+0000","last_became_peered":"2024-09-17T03:15:38.359087+0000","last_unstale":"2024-09-17T03:15:41.051866+0000","last_undegraded":"2024-09-17T03:15:41.051866+0000","last_fullsized":"2024-09-17T03:15:41.051866+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T12:25:28.059613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":13,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.551376+0000","last_change":"2024-09-17T03:15:39.363035+0000","last_active":"2024-09-17T03:15:40.551376+0000","last_peered":"2024-09-17T03:15:40.551376+0000","last_clean":"2024-09-17T03:15:40.551376+0000","last_became_active":"2024-09-17T03:15:39.362744+0000","last_became_peered":"2024-09-17T03:15:39.362744+0000","last_unstale":"2024-09-17T03:15:40.551376+0000","last_undegraded":"2024-09-17T03:15:40.551376+0000","last_fullsized":"2024-09-17T03:15:40.551376+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:38.335805+0000","last_clean_scrub_stamp":"2024-09-17T03:15:38.335805+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T14:19:24.524331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"50'1","reported_seq":16,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:40.351379+0000","last_change":"2024-09-17T03:15:38.355270+0000","last_active":"2024-09-17T03:15:40.351379+0000","last_peered":"2024-09-17T03:15:40.351379+0000","last_clean":"2024-09-17T03:15:40.351379+0000","last_became_active":"2024-09-17T03:15:38.354970+0000","last_became_peered":"2024-09-17T03:15:38.354970+0000","last_unstale":"2024-09-17T03:15:40.351379+0000","last_undegraded":"2024-09-17T03:15:40.351379+0000","last_fullsized":"2024-09-17T03:15:40.351379+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:15:37.333291+0000","last_clean_scrub_stamp":"2024-09-17T03:15:37.333291+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T10:57:42.968414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":159,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-17T03:15:41.051661+0000","last_change":"2024-09-17T03:14:04.862167+0000","last_active":"2024-09-17T03:15:41.051661+0000","last_peered":"2024-09-17T03:15:41.051661+0000","last_clean":"2024-09-17T03:15:41.051661+0000","last_became_active":"2024-09-17T03:14:04.861731+0000","last_became_peered":"2024-09-17T03:14:04.861731+0000","last_unstale":"2024-09-17T03:15:41.051661+0000","last_undegraded":"2024-09-17T03:15:41.051661+0000","last_fullsized":"2024-09-17T03:15:41.051661+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_clean_scrub_stamp":"2024-09-17T03:14:03.847721+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:32.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509465,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27608,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344232,"statfs":{"total":96636764160,"available":96608493568,"internally_reserved":0,"allocated":593920,"data_stored":240686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509465,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":589824,"data_stored":241212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32360,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339480,"statfs":{"total":96636764160,"available":96603627520,"internally_reserved":0,"allocated":1196032,"data_stored":832700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509465,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":589824,"data_stored":240720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509465,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":589824,"data_stored":240664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509465,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1183744,"data_stored":831068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509464,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27604,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344236,"statfs":{"total":96636764160,"available":96608497664,"internally_reserved":0,"allocated":589824,"data_stored":241212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31784,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340056,"statfs":{"total":96636764160,"available":96604217344,"internally_reserved":0,"allocated":606208,"data_stored":242334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:15:47.136 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T03:15:47.136 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-17T03:15:47.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.294+0000 7efc7574a640 1 -- 172.21.15.87:0/4053860280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc700b3e00 msgr2=0x7efc700cfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:47.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.294+0000 7efc7574a640 1 --2- 172.21.15.87:0/4053860280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc700b3e00 0x7efc700cfe60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7efc7008d740 tx=0x7efc6402a5f0 comp rx=0 tx=0).stop 2024-09-17T03:15:47.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.294+0000 7efc7574a640 1 -- 172.21.15.87:0/4053860280 shutdown_connections 2024-09-17T03:15:47.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.294+0000 7efc7574a640 1 --2- 172.21.15.87:0/4053860280 >> v2:172.21.15.149:3300/0 conn(0x7efc700d4350 0x7efc700d4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:47.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.294+0000 7efc7574a640 1 --2- 172.21.15.87:0/4053860280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc700b3e00 0x7efc700cfe60 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:47.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.294+0000 7efc7574a640 1 -- 172.21.15.87:0/4053860280 >> 172.21.15.87:0/4053860280 conn(0x7efc7008bfc0 msgr2=0x7efc7009d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:47.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.294+0000 7efc7574a640 1 -- 172.21.15.87:0/4053860280 shutdown_connections 2024-09-17T03:15:47.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.294+0000 7efc7574a640 1 -- 172.21.15.87:0/4053860280 wait complete. 2024-09-17T03:15:47.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.296+0000 7efc7574a640 1 Processor -- start 2024-09-17T03:15:47.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.296+0000 7efc7574a640 1 -- start start 2024-09-17T03:15:47.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.297+0000 7efc7574a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc700b3080 0x7efc701e70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:47.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.297+0000 7efc7574a640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efc700b3e00 0x7efc70150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:47.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.297+0000 7efc7574a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc700d4350 0x7efc700ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:47.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.297+0000 7efc7574a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efc700d29e0 con 0x7efc700d4350 2024-09-17T03:15:47.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.297+0000 7efc7574a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7efc700d2860 con 0x7efc700b3080 2024-09-17T03:15:47.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.297+0000 7efc7574a640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7efc700d2b60 con 0x7efc700b3e00 2024-09-17T03:15:47.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.297+0000 7efc6f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc700d4350 0x7efc700ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:47.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.297+0000 7efc6f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc700d4350 0x7efc700ca6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42208/0 (socket says 172.21.15.87:42208) 2024-09-17T03:15:47.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.297+0000 7efc6f7fe640 1 -- 172.21.15.87:0/3680379724 learned_addr learned my addr 172.21.15.87:0/3680379724 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:47.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.297+0000 7efc6effd640 1 --2- 172.21.15.87:0/3680379724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc700b3080 0x7efc701e70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:47.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.297+0000 7efc6e7fc640 1 --2- 172.21.15.87:0/3680379724 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efc700b3e00 0x7efc70150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:47.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.297+0000 7efc6f7fe640 1 -- 172.21.15.87:0/3680379724 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efc700b3e00 msgr2=0x7efc70150b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:47.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.297+0000 7efc6f7fe640 1 --2- 172.21.15.87:0/3680379724 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efc700b3e00 0x7efc70150b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:47.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.297+0000 7efc6f7fe640 1 -- 172.21.15.87:0/3680379724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc700b3080 msgr2=0x7efc701e70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:47.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.297+0000 7efc6f7fe640 1 --2- 172.21.15.87:0/3680379724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc700b3080 0x7efc701e70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:47.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.298+0000 7efc6f7fe640 1 -- 172.21.15.87:0/3680379724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc700c9610 con 0x7efc700d4350 2024-09-17T03:15:47.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.298+0000 7efc6f7fe640 1 --2- 172.21.15.87:0/3680379724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc700d4350 0x7efc700ca6b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7efc6800ea80 tx=0x7efc6800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:47.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.298+0000 7efc577fe640 1 -- 172.21.15.87:0/3680379724 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc68004b20 con 0x7efc700d4350 2024-09-17T03:15:47.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.298+0000 7efc577fe640 1 -- 172.21.15.87:0/3680379724 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc68004ce0 con 0x7efc700d4350 2024-09-17T03:15:47.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.298+0000 7efc7574a640 1 -- 172.21.15.87:0/3680379724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc700c98f0 con 0x7efc700d4350 2024-09-17T03:15:47.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.298+0000 7efc577fe640 1 -- 172.21.15.87:0/3680379724 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc68005280 con 0x7efc700d4350 2024-09-17T03:15:47.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.298+0000 7efc7574a640 1 -- 172.21.15.87:0/3680379724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc700c9eb0 con 0x7efc700d4350 2024-09-17T03:15:47.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.299+0000 7efc7574a640 1 -- 172.21.15.87:0/3680379724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc700d3080 con 0x7efc700d4350 2024-09-17T03:15:47.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.300+0000 7efc577fe640 1 -- 172.21.15.87:0/3680379724 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7efc68005420 con 0x7efc700d4350 2024-09-17T03:15:47.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.300+0000 7efc577fe640 1 --2- 172.21.15.87:0/3680379724 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7efc5c078b90 0x7efc5c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:47.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.300+0000 7efc577fe640 1 -- 172.21.15.87:0/3680379724 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7efc6809bb70 con 0x7efc700d4350 2024-09-17T03:15:47.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.301+0000 7efc6effd640 1 --2- 172.21.15.87:0/3680379724 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7efc5c078b90 0x7efc5c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:47.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.301+0000 7efc6effd640 1 --2- 172.21.15.87:0/3680379724 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7efc5c078b90 0x7efc5c07b0a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efc700c7fc0 tx=0x7efc60009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:47.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.304+0000 7efc577fe640 1 -- 172.21.15.87:0/3680379724 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7efc68063b60 con 0x7efc700d4350 2024-09-17T03:15:47.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.549+0000 7efc7574a640 1 -- 172.21.15.87:0/3680379724 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7efc700c93a0 con 0x7efc700d4350 2024-09-17T03:15:47.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.552+0000 7efc577fe640 1 -- 172.21.15.87:0/3680379724 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7efc68068590 con 0x7efc700d4350 2024-09-17T03:15:47.550 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-09-17T03:15:47.551 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDH8+hmLskNLhAAd2M5ynGP8W6Pim11F18wYA== 2024-09-17T03:15:47.551 INFO:teuthology.orchestra.run.smithi087.stdout: caps mds = "allow" 2024-09-17T03:15:47.551 INFO:teuthology.orchestra.run.smithi087.stdout: caps mon = "allow r" 2024-09-17T03:15:47.551 INFO:teuthology.orchestra.run.smithi087.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-17T03:15:47.551 INFO:teuthology.orchestra.run.smithi087.stderr:updated caps for client.0 2024-09-17T03:15:47.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.554+0000 7efc7574a640 1 -- 172.21.15.87:0/3680379724 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7efc5c078b90 msgr2=0x7efc5c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:47.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.554+0000 7efc7574a640 1 --2- 172.21.15.87:0/3680379724 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7efc5c078b90 0x7efc5c07b0a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7efc700c7fc0 tx=0x7efc60009040 comp rx=0 tx=0).stop 2024-09-17T03:15:47.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.554+0000 7efc7574a640 1 -- 172.21.15.87:0/3680379724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc700d4350 msgr2=0x7efc700ca6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:47.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.554+0000 7efc7574a640 1 --2- 172.21.15.87:0/3680379724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc700d4350 0x7efc700ca6b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7efc6800ea80 tx=0x7efc6800ef40 comp rx=0 tx=0).stop 2024-09-17T03:15:47.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.554+0000 7efc7574a640 1 -- 172.21.15.87:0/3680379724 shutdown_connections 2024-09-17T03:15:47.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.554+0000 7efc7574a640 1 --2- 172.21.15.87:0/3680379724 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7efc5c078b90 0x7efc5c07b0a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:47.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.554+0000 7efc7574a640 1 --2- 172.21.15.87:0/3680379724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc700d4350 0x7efc700ca6b0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:47.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.554+0000 7efc7574a640 1 --2- 172.21.15.87:0/3680379724 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7efc700b3e00 0x7efc70150b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:47.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.554+0000 7efc7574a640 1 --2- 172.21.15.87:0/3680379724 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7efc700b3080 0x7efc701e70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:47.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.554+0000 7efc7574a640 1 -- 172.21.15.87:0/3680379724 >> 172.21.15.87:0/3680379724 conn(0x7efc7008bfc0 msgr2=0x7efc7006f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:47.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.554+0000 7efc7574a640 1 -- 172.21.15.87:0/3680379724 shutdown_connections 2024-09-17T03:15:47.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.554+0000 7efc7574a640 1 -- 172.21.15.87:0/3680379724 wait complete. 2024-09-17T03:15:47.565 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:15:47.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.677+0000 7ffa251c6640 1 -- 172.21.15.87:0/1972215115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa200d4860 msgr2=0x7ffa200d3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:47.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.677+0000 7ffa251c6640 1 --2- 172.21.15.87:0/1972215115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa200d4860 0x7ffa200d3060 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ffa14009830 tx=0x7ffa1402a480 comp rx=0 tx=0).stop 2024-09-17T03:15:47.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.677+0000 7ffa251c6640 1 -- 172.21.15.87:0/1972215115 shutdown_connections 2024-09-17T03:15:47.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.677+0000 7ffa251c6640 1 --2- 172.21.15.87:0/1972215115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa200d4860 0x7ffa200d3060 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:47.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.677+0000 7ffa251c6640 1 -- 172.21.15.87:0/1972215115 >> 172.21.15.87:0/1972215115 conn(0x7ffa2008dae0 msgr2=0x7ffa2009d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:47.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.677+0000 7ffa251c6640 1 -- 172.21.15.87:0/1972215115 shutdown_connections 2024-09-17T03:15:47.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.677+0000 7ffa251c6640 1 -- 172.21.15.87:0/1972215115 wait complete. 2024-09-17T03:15:47.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.679+0000 7ffa251c6640 1 Processor -- start 2024-09-17T03:15:47.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.679+0000 7ffa251c6640 1 -- start start 2024-09-17T03:15:47.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.680+0000 7ffa251c6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa200cefb0 0x7ffa2015f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:47.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.680+0000 7ffa251c6640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa200d2c50 0x7ffa201504b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:47.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.680+0000 7ffa251c6640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa200d4860 0x7ffa201509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:47.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.681+0000 7ffa251c6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ffa200cfea0 con 0x7ffa200cefb0 2024-09-17T03:15:47.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.681+0000 7ffa251c6640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ffa200cfd20 con 0x7ffa200d2c50 2024-09-17T03:15:47.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.681+0000 7ffa251c6640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ffa200d0020 con 0x7ffa200d4860 2024-09-17T03:15:47.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.681+0000 7ffa1ed9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa200cefb0 0x7ffa2015f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:47.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.681+0000 7ffa1ed9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa200cefb0 0x7ffa2015f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42230/0 (socket says 172.21.15.87:42230) 2024-09-17T03:15:47.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.681+0000 7ffa1ed9d640 1 -- 172.21.15.87:0/2449577781 learned_addr learned my addr 172.21.15.87:0/2449577781 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:47.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.681+0000 7ffa1e59c640 1 --2- 172.21.15.87:0/2449577781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa200d2c50 0x7ffa201504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:47.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.681+0000 7ffa1f59e640 1 --2- 172.21.15.87:0/2449577781 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa200d4860 0x7ffa201509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:47.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.681+0000 7ffa1ed9d640 1 -- 172.21.15.87:0/2449577781 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa200d4860 msgr2=0x7ffa201509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:47.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.681+0000 7ffa1ed9d640 1 --2- 172.21.15.87:0/2449577781 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa200d4860 0x7ffa201509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:47.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.681+0000 7ffa1ed9d640 1 -- 172.21.15.87:0/2449577781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa200d2c50 msgr2=0x7ffa201504b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:47.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.681+0000 7ffa1ed9d640 1 --2- 172.21.15.87:0/2449577781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa200d2c50 0x7ffa201504b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:47.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.681+0000 7ffa1ed9d640 1 -- 172.21.15.87:0/2449577781 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa20159ed0 con 0x7ffa200cefb0 2024-09-17T03:15:47.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.681+0000 7ffa1ed9d640 1 --2- 172.21.15.87:0/2449577781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa200cefb0 0x7ffa2015f930 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000c3a0 tx=0x7ffa1000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:47.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.681+0000 7ffa077fe640 1 -- 172.21.15.87:0/2449577781 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10011790 con 0x7ffa200cefb0 2024-09-17T03:15:47.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.681+0000 7ffa077fe640 1 -- 172.21.15.87:0/2449577781 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa10017070 con 0x7ffa200cefb0 2024-09-17T03:15:47.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.681+0000 7ffa251c6640 1 -- 172.21.15.87:0/2449577781 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa2015a1b0 con 0x7ffa200cefb0 2024-09-17T03:15:47.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.681+0000 7ffa077fe640 1 -- 172.21.15.87:0/2449577781 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10011cc0 con 0x7ffa200cefb0 2024-09-17T03:15:47.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.681+0000 7ffa251c6640 1 -- 172.21.15.87:0/2449577781 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa2015a770 con 0x7ffa200cefb0 2024-09-17T03:15:47.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.682+0000 7ffa251c6640 1 -- 172.21.15.87:0/2449577781 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa200d3060 con 0x7ffa200cefb0 2024-09-17T03:15:47.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.683+0000 7ffa077fe640 1 -- 172.21.15.87:0/2449577781 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffa1001c460 con 0x7ffa200cefb0 2024-09-17T03:15:47.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.684+0000 7ffa077fe640 1 --2- 172.21.15.87:0/2449577781 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ffa0c078b90 0x7ffa0c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:47.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.684+0000 7ffa077fe640 1 -- 172.21.15.87:0/2449577781 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7ffa1009aa50 con 0x7ffa200cefb0 2024-09-17T03:15:47.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.684+0000 7ffa1e59c640 1 --2- 172.21.15.87:0/2449577781 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ffa0c078b90 0x7ffa0c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:47.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.685+0000 7ffa1e59c640 1 --2- 172.21.15.87:0/2449577781 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ffa0c078b90 0x7ffa0c07b0a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ffa14034000 tx=0x7ffa14004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:47.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.688+0000 7ffa077fe640 1 -- 172.21.15.87:0/2449577781 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ffa100627c0 con 0x7ffa200cefb0 2024-09-17T03:15:47.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.898+0000 7ffa251c6640 1 -- 172.21.15.87:0/2449577781 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa2006e2d0 con 0x7ffa200cefb0 2024-09-17T03:15:47.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.898+0000 7ffa077fe640 1 -- 172.21.15.87:0/2449577781 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5229 (secure 0 0 0) 0x7ffa100671f0 con 0x7ffa200cefb0 2024-09-17T03:15:47.897 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:15:47.897 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":20,"btime":"2024-09-17T03:15:41:361821+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5104,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/1606650259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1606650259},{"type":"v1","addr":"172.21.15.87:6834","nonce":1606650259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":5116,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6836/3876173555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3876173555},{"type":"v1","addr":"172.21.15.149:6836","nonce":3876173555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1657329632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":1657329632},{"type":"v1","addr":"172.21.15.87:6835","nonce":1657329632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:15:38.707286+0000","modified":"2024-09-17T03:15:40.712230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5487":{"gid":5487,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6837/3562056427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6835","nonce":3562056427},{"type":"v1","addr":"172.21.15.149:6837","nonce":3562056427}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5487,"qdb_cluster":[5487]},"id":3}]} 2024-09-17T03:15:47.898 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 20 2024-09-17T03:15:47.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.900+0000 7ffa251c6640 1 -- 172.21.15.87:0/2449577781 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ffa0c078b90 msgr2=0x7ffa0c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:47.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.900+0000 7ffa251c6640 1 --2- 172.21.15.87:0/2449577781 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ffa0c078b90 0x7ffa0c07b0a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ffa14034000 tx=0x7ffa14004730 comp rx=0 tx=0).stop 2024-09-17T03:15:47.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.900+0000 7ffa251c6640 1 -- 172.21.15.87:0/2449577781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa200cefb0 msgr2=0x7ffa2015f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:47.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.900+0000 7ffa251c6640 1 --2- 172.21.15.87:0/2449577781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa200cefb0 0x7ffa2015f930 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ffa1000c3a0 tx=0x7ffa1000c860 comp rx=0 tx=0).stop 2024-09-17T03:15:47.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.900+0000 7ffa251c6640 1 -- 172.21.15.87:0/2449577781 shutdown_connections 2024-09-17T03:15:47.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.900+0000 7ffa251c6640 1 --2- 172.21.15.87:0/2449577781 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ffa0c078b90 0x7ffa0c07b0a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:47.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.900+0000 7ffa251c6640 1 --2- 172.21.15.87:0/2449577781 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ffa200d4860 0x7ffa201509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:47.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.900+0000 7ffa251c6640 1 --2- 172.21.15.87:0/2449577781 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ffa200d2c50 0x7ffa201504b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:47.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.900+0000 7ffa251c6640 1 --2- 172.21.15.87:0/2449577781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffa200cefb0 0x7ffa2015f930 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:47.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.901+0000 7ffa251c6640 1 -- 172.21.15.87:0/2449577781 >> 172.21.15.87:0/2449577781 conn(0x7ffa2008dae0 msgr2=0x7ffa2006dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:47.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.901+0000 7ffa251c6640 1 -- 172.21.15.87:0/2449577781 shutdown_connections 2024-09-17T03:15:47.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:47.901+0000 7ffa251c6640 1 -- 172.21.15.87:0/2449577781 wait complete. 2024-09-17T03:15:48.313 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-17T03:15:38.707286+0000', 'modified': '2024-09-17T03:15:40.712230+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5487}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5487': {'gid': 5487, 'name': 'b', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.149:6837/3562056427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6835', 'nonce': 3562056427}, {'type': 'v1', 'addr': '172.21.15.149:6837', 'nonce': 3562056427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5487, 'qdb_cluster': [5487]} 2024-09-17T03:15:48.313 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-17T03:15:48.313 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-17T03:15:48.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.430+0000 7fa98afc3640 1 -- 172.21.15.87:0/3579242371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa98415e8f0 msgr2=0x7fa98415ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:48.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.430+0000 7fa98afc3640 1 --2- 172.21.15.87:0/3579242371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa98415e8f0 0x7fa98415ed00 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa984071c40 tx=0x7fa97402a4d0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.430+0000 7fa98afc3640 1 -- 172.21.15.87:0/3579242371 shutdown_connections 2024-09-17T03:15:48.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.430+0000 7fa98afc3640 1 --2- 172.21.15.87:0/3579242371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa98415e8f0 0x7fa98415ed00 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.430+0000 7fa98afc3640 1 --2- 172.21.15.87:0/3579242371 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa98415f240 0x7fa9840cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.430+0000 7fa98afc3640 1 --2- 172.21.15.87:0/3579242371 >> v2:172.21.15.149:3300/0 conn(0x7fa9840af680 0x7fa98415acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.430+0000 7fa98afc3640 1 -- 172.21.15.87:0/3579242371 >> 172.21.15.87:0/3579242371 conn(0x7fa98406ee70 msgr2=0x7fa98406f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:48.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.430+0000 7fa98afc3640 1 -- 172.21.15.87:0/3579242371 shutdown_connections 2024-09-17T03:15:48.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.430+0000 7fa98afc3640 1 -- 172.21.15.87:0/3579242371 wait complete. 2024-09-17T03:15:48.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.432+0000 7fa98afc3640 1 Processor -- start 2024-09-17T03:15:48.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.432+0000 7fa98afc3640 1 -- start start 2024-09-17T03:15:48.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.433+0000 7fa98afc3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9840af680 0x7fa98414f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:48.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.433+0000 7fa98afc3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa98415e8f0 0x7fa9840b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:48.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.433+0000 7fa98afc3640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa98415f240 0x7fa9840b1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:48.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.433+0000 7fa98afc3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa9840d01c0 con 0x7fa98415e8f0 2024-09-17T03:15:48.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.433+0000 7fa98afc3640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa9840d0040 con 0x7fa9840af680 2024-09-17T03:15:48.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.433+0000 7fa98afc3640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa9840d0340 con 0x7fa98415f240 2024-09-17T03:15:48.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.433+0000 7fa97bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa98415e8f0 0x7fa9840b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:48.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.433+0000 7fa97bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa98415e8f0 0x7fa9840b1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42240/0 (socket says 172.21.15.87:42240) 2024-09-17T03:15:48.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.433+0000 7fa97bfff640 1 -- 172.21.15.87:0/4147109541 learned_addr learned my addr 172.21.15.87:0/4147109541 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:48.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.434+0000 7fa989560640 1 --2- 172.21.15.87:0/4147109541 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa98415f240 0x7fa9840b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:48.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.434+0000 7fa988d5f640 1 --2- 172.21.15.87:0/4147109541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9840af680 0x7fa98414f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:48.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.434+0000 7fa97bfff640 1 -- 172.21.15.87:0/4147109541 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa98415f240 msgr2=0x7fa9840b1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:48.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.434+0000 7fa97bfff640 1 --2- 172.21.15.87:0/4147109541 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa98415f240 0x7fa9840b1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.434+0000 7fa97bfff640 1 -- 172.21.15.87:0/4147109541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9840af680 msgr2=0x7fa98414f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:48.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.434+0000 7fa97bfff640 1 --2- 172.21.15.87:0/4147109541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9840af680 0x7fa98414f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.434+0000 7fa97bfff640 1 -- 172.21.15.87:0/4147109541 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9840b5ed0 con 0x7fa98415e8f0 2024-09-17T03:15:48.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.434+0000 7fa97bfff640 1 --2- 172.21.15.87:0/4147109541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa98415e8f0 0x7fa9840b1240 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa974034040 tx=0x7fa974003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:48.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.434+0000 7fa9797fa640 1 -- 172.21.15.87:0/4147109541 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97403e030 con 0x7fa98415e8f0 2024-09-17T03:15:48.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.434+0000 7fa9797fa640 1 -- 172.21.15.87:0/4147109541 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa974003e00 con 0x7fa98415e8f0 2024-09-17T03:15:48.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.434+0000 7fa9797fa640 1 -- 172.21.15.87:0/4147109541 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974035040 con 0x7fa98415e8f0 2024-09-17T03:15:48.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.434+0000 7fa98afc3640 1 -- 172.21.15.87:0/4147109541 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9840b5910 con 0x7fa98415e8f0 2024-09-17T03:15:48.435 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.434+0000 7fa98afc3640 1 -- 172.21.15.87:0/4147109541 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9840b09f0 con 0x7fa98415e8f0 2024-09-17T03:15:48.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.436+0000 7fa9797fa640 1 -- 172.21.15.87:0/4147109541 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa974041030 con 0x7fa98415e8f0 2024-09-17T03:15:48.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.437+0000 7fa9797fa640 1 --2- 172.21.15.87:0/4147109541 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa970078b90 0x7fa97007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:48.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.437+0000 7fa9797fa640 1 -- 172.21.15.87:0/4147109541 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fa9740bb4d0 con 0x7fa98415e8f0 2024-09-17T03:15:48.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.437+0000 7fa988d5f640 1 --2- 172.21.15.87:0/4147109541 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa970078b90 0x7fa97007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:48.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.437+0000 7fa992012740 1 Processor -- start 2024-09-17T03:15:48.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.438+0000 7fa988d5f640 1 --2- 172.21.15.87:0/4147109541 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa970078b90 0x7fa97007b0a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa98000daa0 tx=0x7fa980005d70 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:48.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.438+0000 7fa992012740 1 -- start start 2024-09-17T03:15:48.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.439+0000 7fa992012740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d96270 0x562f33167fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:48.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.439+0000 7fa992012740 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562f32d6d3b0 0x562f331684e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:48.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.439+0000 7fa992012740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f32d75590 0x562f331853e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:48.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.439+0000 7fa992012740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x562f330b0820 con 0x562f32d96270 2024-09-17T03:15:48.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.439+0000 7fa992012740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562f3309fc50 con 0x562f32d75590 2024-09-17T03:15:48.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.439+0000 7fa992012740 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x562f330a3a60 con 0x562f32d6d3b0 2024-09-17T03:15:48.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.439+0000 7fa988d5f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d96270 0x562f33167fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:48.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.439+0000 7fa988d5f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d96270 0x562f33167fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42250/0 (socket says 172.21.15.87:42250) 2024-09-17T03:15:48.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.439+0000 7fa988d5f640 1 -- 172.21.15.87:0/1905262377 learned_addr learned my addr 172.21.15.87:0/1905262377 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:48.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.439+0000 7fa989560640 1 --2- 172.21.15.87:0/1905262377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f32d75590 0x562f331853e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:48.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.439+0000 7fa97bfff640 1 --2- 172.21.15.87:0/1905262377 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562f32d6d3b0 0x562f331684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:48.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.439+0000 7fa988d5f640 1 -- 172.21.15.87:0/1905262377 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562f32d6d3b0 msgr2=0x562f331684e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:15:48.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.439+0000 7fa988d5f640 1 --2- 172.21.15.87:0/1905262377 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562f32d6d3b0 0x562f331684e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.439+0000 7fa988d5f640 1 -- 172.21.15.87:0/1905262377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f32d75590 msgr2=0x562f331853e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:15:48.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.439+0000 7fa988d5f640 1 --2- 172.21.15.87:0/1905262377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f32d75590 0x562f331853e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.439+0000 7fa988d5f640 1 -- 172.21.15.87:0/1905262377 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f32e01850 con 0x562f32d96270 2024-09-17T03:15:48.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.439+0000 7fa988d5f640 1 --2- 172.21.15.87:0/1905262377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d96270 0x562f33167fa0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa98000d3b0 tx=0x7fa980012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447e9412925d91b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:48.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.439+0000 7fa96a7fc640 1 -- 172.21.15.87:0/1905262377 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa980022070 con 0x562f32d96270 2024-09-17T03:15:48.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.440+0000 7fa96a7fc640 1 -- 172.21.15.87:0/1905262377 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa98001e450 con 0x562f32d96270 2024-09-17T03:15:48.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.440+0000 7fa96a7fc640 1 -- 172.21.15.87:0/1905262377 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98001e800 con 0x562f32d96270 2024-09-17T03:15:48.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.440+0000 7fa992012740 1 -- 172.21.15.87:0/1905262377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d96270 msgr2=0x562f33167fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:48.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.440+0000 7fa992012740 1 --2- 172.21.15.87:0/1905262377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d96270 0x562f33167fa0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa98000d3b0 tx=0x7fa980012000 comp rx=0 tx=0).stop 2024-09-17T03:15:48.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.440+0000 7fa992012740 1 -- 172.21.15.87:0/1905262377 shutdown_connections 2024-09-17T03:15:48.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.440+0000 7fa992012740 1 --2- 172.21.15.87:0/1905262377 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f32d75590 0x562f331853e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.440+0000 7fa992012740 1 --2- 172.21.15.87:0/1905262377 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562f32d6d3b0 0x562f331684e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.440+0000 7fa992012740 1 --2- 172.21.15.87:0/1905262377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d96270 0x562f33167fa0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.440+0000 7fa992012740 1 -- 172.21.15.87:0/1905262377 >> 172.21.15.87:0/1905262377 conn(0x562f32d8a3c0 msgr2=0x562f3313e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:48.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.440+0000 7fa992012740 1 -- 172.21.15.87:0/1905262377 shutdown_connections 2024-09-17T03:15:48.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.440+0000 7fa992012740 1 -- 172.21.15.87:0/1905262377 wait complete. 2024-09-17T03:15:48.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.441+0000 7fa992012740 1 -- start start 2024-09-17T03:15:48.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.442+0000 7fa992012740 1 Processor -- start 2024-09-17T03:15:48.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.443+0000 7fa992012740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f32d6d3b0 0x562f331961f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:48.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.443+0000 7fa992012740 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562f32d75590 0x562f33191b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:48.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.443+0000 7fa992012740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d96270 0x562f33192060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:48.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.443+0000 7fa992012740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x562f32e10620 con 0x562f32d96270 2024-09-17T03:15:48.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.443+0000 7fa992012740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562f330455f0 con 0x562f32d6d3b0 2024-09-17T03:15:48.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.443+0000 7fa992012740 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x562f33168a20 con 0x562f32d75590 2024-09-17T03:15:48.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.443+0000 7fa992012740 10 client.0 _ll_register_callbacks cb 0x562f330490a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:15:48.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.443+0000 7fa989560640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d96270 0x562f33192060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:48.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.443+0000 7fa989560640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d96270 0x562f33192060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42264/0 (socket says 172.21.15.87:42264) 2024-09-17T03:15:48.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.443+0000 7fa989560640 1 -- 172.21.15.87:0/1991793927 learned_addr learned my addr 172.21.15.87:0/1991793927 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:48.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.443+0000 7fa988d5f640 1 --2- 172.21.15.87:0/1991793927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f32d6d3b0 0x562f331961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:48.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.443+0000 7fa97bfff640 1 --2- 172.21.15.87:0/1991793927 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562f32d75590 0x562f33191b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:48.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.443+0000 7fa989560640 1 -- 172.21.15.87:0/1991793927 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562f32d75590 msgr2=0x562f33191b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:15:48.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.443+0000 7fa989560640 1 --2- 172.21.15.87:0/1991793927 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562f32d75590 0x562f33191b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.443+0000 7fa989560640 1 -- 172.21.15.87:0/1991793927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f32d6d3b0 msgr2=0x562f331961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:15:48.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.443+0000 7fa989560640 1 --2- 172.21.15.87:0/1991793927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f32d6d3b0 0x562f331961f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.443+0000 7fa989560640 1 -- 172.21.15.87:0/1991793927 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f32e786b0 con 0x562f32d96270 2024-09-17T03:15:48.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.443+0000 7fa989560640 1 --2- 172.21.15.87:0/1991793927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d96270 0x562f33192060 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0139f0 tx=0x7fa97c013eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f46e425515ac10 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:48.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.443+0000 7fa9637fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-09-17T03:15:48.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.444+0000 7fa9637fe640 1 -- 172.21.15.87:0/1991793927 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c003970 con 0x562f32d96270 2024-09-17T03:15:48.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.444+0000 7fa9637fe640 1 -- 172.21.15.87:0/1991793927 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fa97c003b10 con 0x562f32d96270 2024-09-17T03:15:48.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.444+0000 7fa992012740 1 -- 172.21.15.87:0/1991793927 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa98001e450 con 0x562f32d96270 2024-09-17T03:15:48.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.444+0000 7fa9637fe640 1 -- 172.21.15.87:0/1991793927 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa97c003d10 con 0x562f32d96270 2024-09-17T03:15:48.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.444+0000 7fa992012740 10 client.5631 fetch_fsmap learned FSMap version 20 2024-09-17T03:15:48.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.444+0000 7fa992012740 1 -- 172.21.15.87:0/1991793927 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x562f33192610 con 0x562f32d96270 2024-09-17T03:15:48.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.444+0000 7fa9637fe640 1 -- 172.21.15.87:0/1991793927 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa98001e450 con 0x562f32d96270 2024-09-17T03:15:48.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.444+0000 7fa9637fe640 1 -- 172.21.15.87:0/1991793927 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) ==== 3502+0+0 (secure 0 0 0) 0x7fa97c027080 con 0x562f32d96270 2024-09-17T03:15:48.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.444+0000 7fa992012740 10 client.5631 fetch_fsmap finished waiting for FSMap version 20 2024-09-17T03:15:48.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.444+0000 7fa992012740 10 client.5631 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-17T03:15:48.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.444+0000 7fa992012740 20 client.5631 populate_metadata read hostname 'smithi087' 2024-09-17T03:15:48.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.444+0000 7fa992012740 1 --2- 172.21.15.87:0/1991793927 >> [v2:172.21.15.149:6835/3562056427,v1:172.21.15.149:6837/3562056427] conn(0x562f32d54e40 0x562f33186330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:48.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.444+0000 7fa992012740 4 client.5631 mds_command: new command op to 5487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-17T03:15:48.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.444+0000 7fa992012740 1 -- 172.21.15.87:0/1991793927 --> [v2:172.21.15.149:6835/3562056427,v1:172.21.15.149:6837/3562056427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f330b2770 con 0x562f32d54e40 2024-09-17T03:15:48.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.444+0000 7fa988d5f640 1 --2- 172.21.15.87:0/1991793927 >> [v2:172.21.15.149:6835/3562056427,v1:172.21.15.149:6837/3562056427] conn(0x562f32d54e40 0x562f33186330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:48.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.445+0000 7fa9637fe640 10 client.5631 ms_handle_connect on v2:172.21.15.149:6835/3562056427 2024-09-17T03:15:48.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.445+0000 7fa988d5f640 1 --2- 172.21.15.87:0/1991793927 >> [v2:172.21.15.149:6835/3562056427,v1:172.21.15.149:6837/3562056427] conn(0x562f32d54e40 0x562f33186330 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d27780637d416fa3 server_cookie=d1a368cbb7dfd4d9 in_seq=0 out_seq=0 2024-09-17T03:15:48.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.446+0000 7fa9637fe640 1 -- 172.21.15.87:0/1991793927 <== mds.0 v2:172.21.15.149:6835/3562056427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562f330b2770 con 0x562f32d54e40 2024-09-17T03:15:48.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.446+0000 7fa9637fe640 10 client.5631 handle_command_reply: tid=0 2024-09-17T03:15:48.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.446+0000 7fa992012740 1 client.5631 shutdown 2024-09-17T03:15:48.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.446+0000 7fa992012740 1 -- 172.21.15.87:0/1991793927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d96270 msgr2=0x562f33192060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:48.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.446+0000 7fa992012740 1 --2- 172.21.15.87:0/1991793927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d96270 0x562f33192060 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0139f0 tx=0x7fa97c013eb0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.446+0000 7fa992012740 1 -- 172.21.15.87:0/1991793927 shutdown_connections 2024-09-17T03:15:48.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.446+0000 7fa992012740 1 --2- 172.21.15.87:0/1991793927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d96270 0x562f33192060 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.446+0000 7fa992012740 1 --2- 172.21.15.87:0/1991793927 >> [v2:172.21.15.149:6835/3562056427,v1:172.21.15.149:6837/3562056427] conn(0x562f32d54e40 0x562f33186330 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.446+0000 7fa992012740 1 --2- 172.21.15.87:0/1991793927 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562f32d75590 0x562f33191b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.446+0000 7fa992012740 1 --2- 172.21.15.87:0/1991793927 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f32d6d3b0 0x562f331961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.446+0000 7fa9637fe640 0 client.5631 ms_handle_reset on v2:172.21.15.149:6835/3562056427 2024-09-17T03:15:48.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.446+0000 7fa992012740 1 -- 172.21.15.87:0/1991793927 >> 172.21.15.87:0/1991793927 conn(0x562f32d8a3c0 msgr2=0x562f3313e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:48.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.446+0000 7fa992012740 1 -- 172.21.15.87:0/1991793927 shutdown_connections 2024-09-17T03:15:48.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.446+0000 7fa992012740 1 -- 172.21.15.87:0/1991793927 wait complete. 2024-09-17T03:15:48.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.446+0000 7fa992012740 20 client.5631 trim_cache size 0 max 16384 2024-09-17T03:15:48.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.464+0000 7fa992012740 1 Processor -- start 2024-09-17T03:15:48.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.464+0000 7fa992012740 1 -- start start 2024-09-17T03:15:48.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.465+0000 7fa992012740 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562f32d6d3b0 0x562f33164a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:48.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.465+0000 7fa992012740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d75590 0x562f3317c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:48.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.465+0000 7fa992012740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f32d96270 0x562f33167fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:48.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.465+0000 7fa992012740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa9840b4b60 con 0x562f32d75590 2024-09-17T03:15:48.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.465+0000 7fa992012740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fa9841f7120 con 0x562f32d96270 2024-09-17T03:15:48.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.465+0000 7fa992012740 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fa9841f7770 con 0x562f32d6d3b0 2024-09-17T03:15:48.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.466+0000 7fa97bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d75590 0x562f3317c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:48.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.466+0000 7fa97bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d75590 0x562f3317c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42280/0 (socket says 172.21.15.87:42280) 2024-09-17T03:15:48.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.466+0000 7fa97bfff640 1 -- 172.21.15.87:0/860909600 learned_addr learned my addr 172.21.15.87:0/860909600 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:48.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.466+0000 7fa989560640 1 --2- 172.21.15.87:0/860909600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f32d96270 0x562f33167fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:48.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.466+0000 7fa988d5f640 1 --2- 172.21.15.87:0/860909600 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562f32d6d3b0 0x562f33164a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:48.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.466+0000 7fa97bfff640 1 -- 172.21.15.87:0/860909600 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562f32d6d3b0 msgr2=0x562f33164a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:15:48.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.466+0000 7fa97bfff640 1 --2- 172.21.15.87:0/860909600 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562f32d6d3b0 0x562f33164a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.466+0000 7fa97bfff640 1 -- 172.21.15.87:0/860909600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f32d96270 msgr2=0x562f33167fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:15:48.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.466+0000 7fa97bfff640 1 --2- 172.21.15.87:0/860909600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f32d96270 0x562f33167fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.466+0000 7fa97bfff640 1 -- 172.21.15.87:0/860909600 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f3317dc10 con 0x562f32d75590 2024-09-17T03:15:48.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.466+0000 7fa97bfff640 1 --2- 172.21.15.87:0/860909600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d75590 0x562f3317c310 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa974083250 tx=0x7fa9740bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6365eea5e524e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:48.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.466+0000 7fa969ffb640 1 -- 172.21.15.87:0/860909600 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974048040 con 0x562f32d75590 2024-09-17T03:15:48.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.466+0000 7fa969ffb640 1 -- 172.21.15.87:0/860909600 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9740a9030 con 0x562f32d75590 2024-09-17T03:15:48.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.466+0000 7fa969ffb640 1 -- 172.21.15.87:0/860909600 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa974083780 con 0x562f32d75590 2024-09-17T03:15:48.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.467+0000 7fa992012740 1 -- 172.21.15.87:0/860909600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d75590 msgr2=0x562f3317c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:48.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.467+0000 7fa992012740 1 --2- 172.21.15.87:0/860909600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d75590 0x562f3317c310 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa974083250 tx=0x7fa9740bb040 comp rx=0 tx=0).stop 2024-09-17T03:15:48.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.467+0000 7fa992012740 1 -- 172.21.15.87:0/860909600 shutdown_connections 2024-09-17T03:15:48.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.467+0000 7fa992012740 1 --2- 172.21.15.87:0/860909600 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f32d96270 0x562f33167fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.467+0000 7fa992012740 1 --2- 172.21.15.87:0/860909600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d75590 0x562f3317c310 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.467+0000 7fa992012740 1 --2- 172.21.15.87:0/860909600 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562f32d6d3b0 0x562f33164a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.467+0000 7fa992012740 1 -- 172.21.15.87:0/860909600 >> 172.21.15.87:0/860909600 conn(0x562f32d8a3c0 msgr2=0x562f33139ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:48.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.467+0000 7fa992012740 1 -- 172.21.15.87:0/860909600 shutdown_connections 2024-09-17T03:15:48.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.467+0000 7fa992012740 1 -- 172.21.15.87:0/860909600 wait complete. 2024-09-17T03:15:48.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.468+0000 7fa992012740 1 -- start start 2024-09-17T03:15:48.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.468+0000 7fa992012740 1 Processor -- start 2024-09-17T03:15:48.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.469+0000 7fa992012740 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f32d6d3b0 0x562f33205bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:48.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.469+0000 7fa992012740 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562f32d75590 0x562f331ffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:48.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.469+0000 7fa992012740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d96270 0x562f33167d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:48.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.469+0000 7fa992012740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x562f33185610 con 0x562f32d96270 2024-09-17T03:15:48.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.469+0000 7fa992012740 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x562f3317bbf0 con 0x562f32d6d3b0 2024-09-17T03:15:48.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.469+0000 7fa992012740 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x562f3317e1d0 con 0x562f32d75590 2024-09-17T03:15:48.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.469+0000 7fa992012740 10 client.0 _ll_register_callbacks cb 0x7fa97c009af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-17T03:15:48.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.469+0000 7fa989560640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d96270 0x562f33167d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:48.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.469+0000 7fa989560640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d96270 0x562f33167d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42288/0 (socket says 172.21.15.87:42288) 2024-09-17T03:15:48.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.469+0000 7fa989560640 1 -- 172.21.15.87:0/858048404 learned_addr learned my addr 172.21.15.87:0/858048404 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:48.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.469+0000 7fa988d5f640 1 --2- 172.21.15.87:0/858048404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f32d6d3b0 0x562f33205bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:48.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.469+0000 7fa97bfff640 1 --2- 172.21.15.87:0/858048404 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562f32d75590 0x562f331ffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:48.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.470+0000 7fa989560640 1 -- 172.21.15.87:0/858048404 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562f32d75590 msgr2=0x562f331ffbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:15:48.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.470+0000 7fa989560640 1 --2- 172.21.15.87:0/858048404 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562f32d75590 0x562f331ffbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.470+0000 7fa989560640 1 -- 172.21.15.87:0/858048404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f32d6d3b0 msgr2=0x562f33205bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-17T03:15:48.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.470+0000 7fa989560640 1 --2- 172.21.15.87:0/858048404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f32d6d3b0 0x562f33205bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.470+0000 7fa989560640 1 -- 172.21.15.87:0/858048404 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f331a6410 con 0x562f32d96270 2024-09-17T03:15:48.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.470+0000 7fa989560640 1 --2- 172.21.15.87:0/858048404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d96270 0x562f33167d90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0139f0 tx=0x7fa97c003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce9485dfb77c6f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:48.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.470+0000 7fa9637fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-09-17T03:15:48.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.470+0000 7fa9637fe640 1 -- 172.21.15.87:0/858048404 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c004d30 con 0x562f32d96270 2024-09-17T03:15:48.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.470+0000 7fa992012740 1 -- 172.21.15.87:0/858048404 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9740a9030 con 0x562f32d96270 2024-09-17T03:15:48.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.470+0000 7fa9637fe640 1 -- 172.21.15.87:0/858048404 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fa97c01c440 con 0x562f32d96270 2024-09-17T03:15:48.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.470+0000 7fa9637fe640 1 -- 172.21.15.87:0/858048404 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa97c038050 con 0x562f32d96270 2024-09-17T03:15:48.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.470+0000 7fa992012740 10 client.5637 fetch_fsmap learned FSMap version 20 2024-09-17T03:15:48.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.470+0000 7fa992012740 1 -- 172.21.15.87:0/858048404 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) -- 0x562f33191e20 con 0x562f32d96270 2024-09-17T03:15:48.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.470+0000 7fa9637fe640 1 -- 172.21.15.87:0/858048404 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9740a9030 con 0x562f32d96270 2024-09-17T03:15:48.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.470+0000 7fa9637fe640 1 -- 172.21.15.87:0/858048404 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 20) ==== 3502+0+0 (secure 0 0 0) 0x7fa97c03b020 con 0x562f32d96270 2024-09-17T03:15:48.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.470+0000 7fa992012740 10 client.5637 fetch_fsmap finished waiting for FSMap version 20 2024-09-17T03:15:48.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.471+0000 7fa992012740 10 client.5637 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-17T03:15:48.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.471+0000 7fa992012740 20 client.5637 populate_metadata read hostname 'smithi087' 2024-09-17T03:15:48.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.471+0000 7fa992012740 1 --2- 172.21.15.87:0/858048404 >> [v2:172.21.15.149:6835/3562056427,v1:172.21.15.149:6837/3562056427] conn(0x562f33209d30 0x562f33190340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:48.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.471+0000 7fa992012740 4 client.5637 mds_command: new command op to 5487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-17T03:15:48.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.471+0000 7fa992012740 1 -- 172.21.15.87:0/858048404 --> [v2:172.21.15.149:6835/3562056427,v1:172.21.15.149:6837/3562056427] -- command(tid 0: {"prefix": "status"}) -- 0x562f330ae220 con 0x562f33209d30 2024-09-17T03:15:48.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.471+0000 7fa988d5f640 1 --2- 172.21.15.87:0/858048404 >> [v2:172.21.15.149:6835/3562056427,v1:172.21.15.149:6837/3562056427] conn(0x562f33209d30 0x562f33190340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:48.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.471+0000 7fa9637fe640 10 client.5637 ms_handle_connect on v2:172.21.15.149:6835/3562056427 2024-09-17T03:15:48.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.471+0000 7fa988d5f640 1 --2- 172.21.15.87:0/858048404 >> [v2:172.21.15.149:6835/3562056427,v1:172.21.15.149:6837/3562056427] conn(0x562f33209d30 0x562f33190340 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f3db80efd083937 server_cookie=b962d00b2708d12d in_seq=0 out_seq=0 2024-09-17T03:15:48.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.471+0000 7fa9637fe640 1 -- 172.21.15.87:0/858048404 <== mds.0 v2:172.21.15.149:6835/3562056427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x562f330ae220 con 0x562f33209d30 2024-09-17T03:15:48.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.471+0000 7fa9637fe640 10 client.5637 handle_command_reply: tid=0 2024-09-17T03:15:48.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.471+0000 7fa992012740 1 client.5637 shutdown 2024-09-17T03:15:48.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.472+0000 7fa992012740 1 -- 172.21.15.87:0/858048404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d96270 msgr2=0x562f33167d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:48.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.472+0000 7fa992012740 1 --2- 172.21.15.87:0/858048404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d96270 0x562f33167d90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa97c0139f0 tx=0x7fa97c003ca0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.472+0000 7fa992012740 1 -- 172.21.15.87:0/858048404 shutdown_connections 2024-09-17T03:15:48.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.472+0000 7fa992012740 1 --2- 172.21.15.87:0/858048404 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562f32d96270 0x562f33167d90 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.472+0000 7fa992012740 1 --2- 172.21.15.87:0/858048404 >> [v2:172.21.15.149:6835/3562056427,v1:172.21.15.149:6837/3562056427] conn(0x562f33209d30 0x562f33190340 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.472+0000 7fa992012740 1 --2- 172.21.15.87:0/858048404 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x562f32d75590 0x562f331ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.472+0000 7fa992012740 1 --2- 172.21.15.87:0/858048404 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x562f32d6d3b0 0x562f33205bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.472+0000 7fa9637fe640 0 client.5637 ms_handle_reset on v2:172.21.15.149:6835/3562056427 2024-09-17T03:15:48.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.472+0000 7fa992012740 1 -- 172.21.15.87:0/858048404 >> 172.21.15.87:0/858048404 conn(0x562f32d8a3c0 msgr2=0x562f331859b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:48.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.472+0000 7fa992012740 1 -- 172.21.15.87:0/858048404 shutdown_connections 2024-09-17T03:15:48.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.472+0000 7fa992012740 1 -- 172.21.15.87:0/858048404 wait complete. 2024-09-17T03:15:48.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.472+0000 7fa992012740 20 client.5637 trim_cache size 0 max 16384 2024-09-17T03:15:48.476 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-17T03:15:48.476 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "7b2553aa-e562-4da8-b175-8e91f460da55", 2024-09-17T03:15:48.476 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-09-17T03:15:48.476 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 5487, 2024-09-17T03:15:48.476 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-09-17T03:15:48.476 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-09-17T03:15:48.476 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-09-17T03:15:48.476 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 9.1266610939999993, 2024-09-17T03:15:48.476 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 20, 2024-09-17T03:15:48.476 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 50, 2024-09-17T03:15:48.476 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 50, 2024-09-17T03:15:48.476 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 14.412876171000001 2024-09-17T03:15:48.477 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-17T03:15:48.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.472+0000 7fa98afc3640 1 -- 172.21.15.87:0/4147109541 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa970078b90 msgr2=0x7fa97007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:48.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.472+0000 7fa98afc3640 1 --2- 172.21.15.87:0/4147109541 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa970078b90 0x7fa97007b0a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa98000daa0 tx=0x7fa980005d70 comp rx=0 tx=0).stop 2024-09-17T03:15:48.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.472+0000 7fa98afc3640 1 -- 172.21.15.87:0/4147109541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa98415e8f0 msgr2=0x7fa9840b1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:48.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.472+0000 7fa98afc3640 1 --2- 172.21.15.87:0/4147109541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa98415e8f0 0x7fa9840b1240 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa974034040 tx=0x7fa974003910 comp rx=0 tx=0).stop 2024-09-17T03:15:48.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.472+0000 7fa98afc3640 1 -- 172.21.15.87:0/4147109541 shutdown_connections 2024-09-17T03:15:48.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.472+0000 7fa98afc3640 1 --2- 172.21.15.87:0/4147109541 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fa970078b90 0x7fa97007b0a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.472+0000 7fa98afc3640 1 --2- 172.21.15.87:0/4147109541 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fa98415f240 0x7fa9840b1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.472+0000 7fa98afc3640 1 --2- 172.21.15.87:0/4147109541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa98415e8f0 0x7fa9840b1240 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.472+0000 7fa98afc3640 1 --2- 172.21.15.87:0/4147109541 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fa9840af680 0x7fa98414f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.472+0000 7fa98afc3640 1 -- 172.21.15.87:0/4147109541 >> 172.21.15.87:0/4147109541 conn(0x7fa98406ee70 msgr2=0x7fa9841564e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:48.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.472+0000 7fa98afc3640 1 -- 172.21.15.87:0/4147109541 shutdown_connections 2024-09-17T03:15:48.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.472+0000 7fa98afc3640 1 -- 172.21.15.87:0/4147109541 wait complete. 2024-09-17T03:15:48.482 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T03:15:48.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.595+0000 7f1278aa2640 1 -- 172.21.15.87:0/439605872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f125c004800 msgr2=0x7f125c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:48.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.595+0000 7f1278aa2640 1 --2- 172.21.15.87:0/439605872 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f125c004800 0x7f125c004c10 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f1274072cc0 tx=0x7f125c005830 comp rx=0 tx=0).stop 2024-09-17T03:15:48.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.595+0000 7f1272d9d640 1 -- 172.21.15.87:0/439605872 reap_dead start 2024-09-17T03:15:48.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.595+0000 7f1278aa2640 1 -- 172.21.15.87:0/439605872 shutdown_connections 2024-09-17T03:15:48.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.595+0000 7f1278aa2640 1 -- 172.21.15.87:0/439605872 >> 172.21.15.87:0/439605872 conn(0x7f127406ee70 msgr2=0x7f127406f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:48.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.596+0000 7f1278aa2640 1 -- 172.21.15.87:0/439605872 shutdown_connections 2024-09-17T03:15:48.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.596+0000 7f1278aa2640 1 -- 172.21.15.87:0/439605872 wait complete. 2024-09-17T03:15:48.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.597+0000 7f1278aa2640 1 Processor -- start 2024-09-17T03:15:48.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.597+0000 7f1278aa2640 1 -- start start 2024-09-17T03:15:48.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.599+0000 7f1278aa2640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f125c004800 0x7f12740b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:48.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.599+0000 7f1278aa2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12740c9740 0x7f1274148400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:48.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.599+0000 7f1278aa2640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1274148940 0x7f12740b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:48.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.599+0000 7f1278aa2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1274073940 con 0x7f12740c9740 2024-09-17T03:15:48.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.599+0000 7f1278aa2640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f12740c5b60 con 0x7f1274148940 2024-09-17T03:15:48.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.599+0000 7f1278aa2640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1274073ac0 con 0x7f125c004800 2024-09-17T03:15:48.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.599+0000 7f1271d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12740c9740 0x7f1274148400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:48.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.599+0000 7f1271d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12740c9740 0x7f1274148400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42300/0 (socket says 172.21.15.87:42300) 2024-09-17T03:15:48.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.599+0000 7f1271d9b640 1 -- 172.21.15.87:0/2710449250 learned_addr learned my addr 172.21.15.87:0/2710449250 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:48.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.599+0000 7f127259c640 1 --2- 172.21.15.87:0/2710449250 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f125c004800 0x7f12740b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:48.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.599+0000 7f1272d9d640 1 --2- 172.21.15.87:0/2710449250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1274148940 0x7f12740b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:48.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.599+0000 7f1271d9b640 1 -- 172.21.15.87:0/2710449250 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f125c004800 msgr2=0x7f12740b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:48.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.599+0000 7f1271d9b640 1 --2- 172.21.15.87:0/2710449250 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f125c004800 0x7f12740b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.599+0000 7f1271d9b640 1 -- 172.21.15.87:0/2710449250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1274148940 msgr2=0x7f12740b3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:48.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.599+0000 7f1271d9b640 1 --2- 172.21.15.87:0/2710449250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1274148940 0x7f12740b3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.599+0000 7f1271d9b640 1 -- 172.21.15.87:0/2710449250 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1274148d50 con 0x7f12740c9740 2024-09-17T03:15:48.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.599+0000 7f1272d9d640 1 --2- 172.21.15.87:0/2710449250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1274148940 0x7f12740b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:48.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.599+0000 7f1271d9b640 1 --2- 172.21.15.87:0/2710449250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12740c9740 0x7f1274148400 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1268000c00 tx=0x7f1268001da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:48.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.599+0000 7f1262ffd640 1 -- 172.21.15.87:0/2710449250 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1268032030 con 0x7f12740c9740 2024-09-17T03:15:48.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.600+0000 7f1262ffd640 1 -- 172.21.15.87:0/2710449250 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12680038e0 con 0x7f12740c9740 2024-09-17T03:15:48.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.600+0000 7f1278aa2640 1 -- 172.21.15.87:0/2710449250 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12741f6f30 con 0x7f12740c9740 2024-09-17T03:15:48.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.600+0000 7f1262ffd640 1 -- 172.21.15.87:0/2710449250 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1268003d30 con 0x7f12740c9740 2024-09-17T03:15:48.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.600+0000 7f1278aa2640 1 -- 172.21.15.87:0/2710449250 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f127414a5d0 con 0x7f12740c9740 2024-09-17T03:15:48.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.601+0000 7f1278aa2640 1 -- 172.21.15.87:0/2710449250 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1274073230 con 0x7f12740c9740 2024-09-17T03:15:48.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.601+0000 7f1262ffd640 1 -- 172.21.15.87:0/2710449250 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1268035020 con 0x7f12740c9740 2024-09-17T03:15:48.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.602+0000 7f1262ffd640 1 --2- 172.21.15.87:0/2710449250 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1254078bc0 0x7f125407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:48.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.602+0000 7f1262ffd640 1 -- 172.21.15.87:0/2710449250 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f12680bb220 con 0x7f12740c9740 2024-09-17T03:15:48.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.602+0000 7f127259c640 1 --2- 172.21.15.87:0/2710449250 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1254078bc0 0x7f125407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:48.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.603+0000 7f127259c640 1 --2- 172.21.15.87:0/2710449250 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1254078bc0 0x7f125407b0d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1264009b50 tx=0x7f1264005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:48.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.606+0000 7f1262ffd640 1 -- 172.21.15.87:0/2710449250 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f1268047030 con 0x7f12740c9740 2024-09-17T03:15:48.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.816+0000 7f1278aa2640 1 -- 172.21.15.87:0/2710449250 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f12740cd600 con 0x7f12740c9740 2024-09-17T03:15:48.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.816+0000 7f1262ffd640 1 -- 172.21.15.87:0/2710449250 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) ==== 53+0+73 (secure 0 0 0) 0x7f1274148d50 con 0x7f12740c9740 2024-09-17T03:15:48.816 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T03:15:48.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.818+0000 7f1278aa2640 1 -- 172.21.15.87:0/2710449250 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1254078bc0 msgr2=0x7f125407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:48.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.818+0000 7f1278aa2640 1 --2- 172.21.15.87:0/2710449250 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1254078bc0 0x7f125407b0d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1264009b50 tx=0x7f1264005f90 comp rx=0 tx=0).stop 2024-09-17T03:15:48.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.818+0000 7f1278aa2640 1 -- 172.21.15.87:0/2710449250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12740c9740 msgr2=0x7f1274148400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:48.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.818+0000 7f1278aa2640 1 --2- 172.21.15.87:0/2710449250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12740c9740 0x7f1274148400 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f1268000c00 tx=0x7f1268001da0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.818+0000 7f1278aa2640 1 -- 172.21.15.87:0/2710449250 shutdown_connections 2024-09-17T03:15:48.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.818+0000 7f1278aa2640 1 --2- 172.21.15.87:0/2710449250 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1254078bc0 0x7f125407b0d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.818+0000 7f1278aa2640 1 --2- 172.21.15.87:0/2710449250 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f1274148940 0x7f12740b3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.818+0000 7f1278aa2640 1 --2- 172.21.15.87:0/2710449250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12740c9740 0x7f1274148400 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.818+0000 7f1278aa2640 1 --2- 172.21.15.87:0/2710449250 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f125c004800 0x7f12740b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:48.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.818+0000 7f1278aa2640 1 -- 172.21.15.87:0/2710449250 >> 172.21.15.87:0/2710449250 conn(0x7f127406ee70 msgr2=0x7f12740d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:48.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.818+0000 7f1278aa2640 1 -- 172.21.15.87:0/2710449250 shutdown_connections 2024-09-17T03:15:48.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:48.819+0000 7f1278aa2640 1 -- 172.21.15.87:0/2710449250 wait complete. 2024-09-17T03:15:48.829 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T03:15:48.829 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T03:15:48.829 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-09-17T03:15:48.829 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T03:15:48.829 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:48.829 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T03:15:48.829 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T03:15:48.829 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T03:15:48.829 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:15:48.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:48.830 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-09-17T03:15:48.845 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:15:48.846 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:15:48.846 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:15:48.846 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:48.846 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-09-17T03:15:48.846 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:48.846 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:15:48.846 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-09-17T03:15:48.846 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-09-17T03:15:48.846 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:15:48.846 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-09-17T03:15:48.847 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-09-17T03:15:48.847 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:15:48.847 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:15:48.847 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T03:15:48.847 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:15:48.847 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T03:15:48.847 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42683sec preferred_lft 42683sec 2024-09-17T03:15:48.847 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link noprefixroute 2024-09-17T03:15:48.848 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:48.848 INFO:teuthology.orchestra.run.smithi087.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:15:48.848 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 26:32:f7:da:05:b7 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:15:48.848 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T03:15:48.848 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:48.848 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c822:ceff:fef0:4a99/64 scope link 2024-09-17T03:15:48.848 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:48.848 INFO:teuthology.orchestra.run.smithi087.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:15:48.848 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether b2:cb:c1:94:14:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:15:48.849 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::b0cb:c1ff:fe94:145d/64 scope link 2024-09-17T03:15:48.849 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:48.849 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:15:48.849 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 26:32:f7:da:05:b7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:15:48.849 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::2432:f7ff:feda:5b7/64 scope link 2024-09-17T03:15:48.849 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:48.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:48.849 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T03:15:48.909 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-09-17T03:15:48.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:48.911 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-09-17T03:15:48.968 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-09-17T03:15:48.968 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T03:15:48.968 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T03:15:48.969 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T03:15:48.969 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T03:15:48.969 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:15:48.969 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-09-17T03:15:48.969 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T03:15:48.969 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T03:15:48.969 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T03:15:48.969 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-09-17T03:15:49.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:15:49.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:15:49.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:49.094 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-09-17T03:15:49.109 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T03:15:49.109 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T03:15:49.110 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:15:49.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:49.110 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-09-17T03:15:49.174 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T03:15:49.175 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:15:49.175 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:49.230 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:15:49.230 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:15:49.230 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:49.285 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-09-17T03:15:49.351 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T03:15:49.408 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-09-17T03:15:49.408 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-09-17T03:15:49.408 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-09-17T03:15:49.408 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-09-17T03:15:49.409 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-09-17T03:15:49.409 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-09-17T03:15:49.409 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-09-17T03:15:49.409 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-09-17T03:15:49.409 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-09-17T03:15:49.409 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-09-17T03:15:49.409 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-09-17T03:15:49.409 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-09-17T03:15:49.409 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T03:15:49.409 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-09-17T03:15:49.409 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-09-17T03:15:49.409 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-09-17T03:15:49.409 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-09-17T03:15:49.410 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-09-17T03:15:49.410 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-09-17T03:15:49.410 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T03:15:49.410 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T03:15:49.410 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T03:15:49.410 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T03:15:49.410 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-09-17T03:15:49.410 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-09-17T03:15:49.410 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T03:15:49.410 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T03:15:49.410 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T03:15:49.410 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T03:15:49.410 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-09-17T03:15:49.410 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:15:49.411 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:15:49.411 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:15:49.411 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:15:49.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:49.411 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-09-17T03:15:49.463 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T03:15:49.464 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-17T03:15:49.506 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-09-17T03:15:49.530 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T03:15:49.588 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-09-17T03:15:49.588 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-09-17T03:15:49.588 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-09-17T03:15:49.588 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-09-17T03:15:49.588 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-09-17T03:15:49.588 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-09-17T03:15:49.588 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-09-17T03:15:49.589 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-09-17T03:15:49.589 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-09-17T03:15:49.589 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-09-17T03:15:49.589 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-09-17T03:15:49.589 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-09-17T03:15:49.589 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T03:15:49.589 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-09-17T03:15:49.589 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-09-17T03:15:49.589 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-09-17T03:15:49.589 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-09-17T03:15:49.589 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-09-17T03:15:49.589 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-09-17T03:15:49.589 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T03:15:49.590 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T03:15:49.590 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T03:15:49.590 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T03:15:49.590 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-09-17T03:15:49.590 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-09-17T03:15:49.590 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T03:15:49.591 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T03:15:49.591 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T03:15:49.591 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T03:15:49.591 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-09-17T03:15:49.591 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:15:49.591 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:15:49.591 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:15:49.591 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:15:49.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:49.591 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-09-17T03:15:49.597 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:2024-09-17T03:15:49.599+0000 7f9fdb947480 -1 init, newargv = 0x7f9fc0001080 newargc=15 2024-09-17T03:15:49.598 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:ceph-fuse[53555]: starting ceph client 2024-09-17T03:15:49.602 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:ceph-fuse[53555]: starting fuse 2024-09-17T03:15:49.612 INFO:teuthology.orchestra.run.smithi087.stdout:42 2024-09-17T03:15:49.612 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-17T03:15:49.613 DEBUG:teuthology.orchestra.run.smithi087:> sudo stdin-killer -- python3 -c ' 2024-09-17T03:15:49.613 DEBUG:teuthology.orchestra.run.smithi087:> import glob 2024-09-17T03:15:49.613 DEBUG:teuthology.orchestra.run.smithi087:> import re 2024-09-17T03:15:49.613 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-09-17T03:15:49.613 DEBUG:teuthology.orchestra.run.smithi087:> import subprocess 2024-09-17T03:15:49.613 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:15:49.613 DEBUG:teuthology.orchestra.run.smithi087:> def _find_admin_socket(client_name): 2024-09-17T03:15:49.613 DEBUG:teuthology.orchestra.run.smithi087:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T03:15:49.613 DEBUG:teuthology.orchestra.run.smithi087:> files = glob.glob(asok_path) 2024-09-17T03:15:49.613 DEBUG:teuthology.orchestra.run.smithi087:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T03:15:49.613 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:15:49.613 DEBUG:teuthology.orchestra.run.smithi087:> # Given a non-glob path, it better be there 2024-09-17T03:15:49.613 DEBUG:teuthology.orchestra.run.smithi087:> if "*" not in asok_path: 2024-09-17T03:15:49.614 DEBUG:teuthology.orchestra.run.smithi087:> assert(len(files) == 1) 2024-09-17T03:15:49.614 DEBUG:teuthology.orchestra.run.smithi087:> return files[0] 2024-09-17T03:15:49.614 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:15:49.614 DEBUG:teuthology.orchestra.run.smithi087:> for f in files: 2024-09-17T03:15:49.614 DEBUG:teuthology.orchestra.run.smithi087:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T03:15:49.614 DEBUG:teuthology.orchestra.run.smithi087:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T03:15:49.614 DEBUG:teuthology.orchestra.run.smithi087:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T03:15:49.614 DEBUG:teuthology.orchestra.run.smithi087:> contents = proc_f.read() 2024-09-17T03:15:49.614 DEBUG:teuthology.orchestra.run.smithi087:> if mountpoint in contents: 2024-09-17T03:15:49.614 DEBUG:teuthology.orchestra.run.smithi087:> return f 2024-09-17T03:15:49.614 DEBUG:teuthology.orchestra.run.smithi087:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T03:15:49.614 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:15:49.614 DEBUG:teuthology.orchestra.run.smithi087:> print(_find_admin_socket("client.0")) 2024-09-17T03:15:49.614 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-09-17T03:15:49.732 INFO:teuthology.orchestra.run.smithi087.stdout:/var/run/ceph/ceph-client.0.53555.asok 2024-09-17T03:15:49.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:15:49.741 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.53555.asok 2024-09-17T03:15:49.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:49.741 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.53555.asok status 2024-09-17T03:15:49.882 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-17T03:15:49.882 INFO:teuthology.orchestra.run.smithi087.stdout: "metadata": { 2024-09-17T03:15:49.882 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T03:15:49.882 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T03:15:49.882 INFO:teuthology.orchestra.run.smithi087.stdout: "entity_id": "0", 2024-09-17T03:15:49.883 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": "smithi087", 2024-09-17T03:15:49.883 INFO:teuthology.orchestra.run.smithi087.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T03:15:49.883 INFO:teuthology.orchestra.run.smithi087.stdout: "pid": "53555", 2024-09-17T03:15:49.883 INFO:teuthology.orchestra.run.smithi087.stdout: "root": "/" 2024-09-17T03:15:49.883 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:49.883 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_count": 0, 2024-09-17T03:15:49.883 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_pinned_count": 0, 2024-09-17T03:15:49.883 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 5649, 2024-09-17T03:15:49.883 INFO:teuthology.orchestra.run.smithi087.stdout: "inst": { 2024-09-17T03:15:49.883 INFO:teuthology.orchestra.run.smithi087.stdout: "name": { 2024-09-17T03:15:49.883 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "client", 2024-09-17T03:15:49.883 INFO:teuthology.orchestra.run.smithi087.stdout: "num": 5649 2024-09-17T03:15:49.883 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:49.883 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-09-17T03:15:49.884 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-09-17T03:15:49.884 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-09-17T03:15:49.884 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 3684651207 2024-09-17T03:15:49.884 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:49.884 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:49.884 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-09-17T03:15:49.884 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-09-17T03:15:49.884 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-09-17T03:15:49.884 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 3684651207 2024-09-17T03:15:49.884 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:49.884 INFO:teuthology.orchestra.run.smithi087.stdout: "inst_str": "client.5649 192.168.0.1:0/3684651207", 2024-09-17T03:15:49.884 INFO:teuthology.orchestra.run.smithi087.stdout: "addr_str": "192.168.0.1:0/3684651207", 2024-09-17T03:15:49.885 INFO:teuthology.orchestra.run.smithi087.stdout: "inode_count": 1, 2024-09-17T03:15:49.885 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_epoch": 20, 2024-09-17T03:15:49.885 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch": 50, 2024-09-17T03:15:49.885 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch_barrier": 0, 2024-09-17T03:15:49.885 INFO:teuthology.orchestra.run.smithi087.stdout: "blocklisted": false, 2024-09-17T03:15:49.885 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs" 2024-09-17T03:15:49.885 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-17T03:15:49.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:49.891 DEBUG:teuthology.orchestra.run.smithi087:> stat --file-system '--printf=%T 2024-09-17T03:15:49.891 DEBUG:teuthology.orchestra.run.smithi087:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:49.907 INFO:teuthology.orchestra.run.smithi087.stdout:fuseblk 2024-09-17T03:15:49.907 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:49.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:49.907 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:49.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:49.974 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch test_file) 2024-09-17T03:15:50.036 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow r pool=cephfs_data' 2024-09-17T03:15:50.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.197+0000 7fcde7a63640 1 -- 172.21.15.87:0/3551054645 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcde0074750 msgr2=0x7fcde006cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:50.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.197+0000 7fcde7a63640 1 --2- 172.21.15.87:0/3551054645 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcde0074750 0x7fcde006cde0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fcde00755c0 tx=0x7fcdd002a560 comp rx=0 tx=0).stop 2024-09-17T03:15:50.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.197+0000 7fcde7a63640 1 -- 172.21.15.87:0/3551054645 shutdown_connections 2024-09-17T03:15:50.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.197+0000 7fcde7a63640 1 --2- 172.21.15.87:0/3551054645 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcde0074750 0x7fcde006cde0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:50.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.197+0000 7fcde7a63640 1 --2- 172.21.15.87:0/3551054645 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcde0074b60 0x7fcde006d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:50.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.197+0000 7fcde7a63640 1 --2- 172.21.15.87:0/3551054645 >> v2:172.21.15.149:3300/0 conn(0x7fcde0162f40 0x7fcde0073e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:50.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.197+0000 7fcde7a63640 1 -- 172.21.15.87:0/3551054645 >> 172.21.15.87:0/3551054645 conn(0x7fcde00cdc80 msgr2=0x7fcde00ce090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:50.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.197+0000 7fcde7a63640 1 -- 172.21.15.87:0/3551054645 shutdown_connections 2024-09-17T03:15:50.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.197+0000 7fcde7a63640 1 -- 172.21.15.87:0/3551054645 wait complete. 2024-09-17T03:15:50.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.199+0000 7fcde7a63640 1 Processor -- start 2024-09-17T03:15:50.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.199+0000 7fcde7a63640 1 -- start start 2024-09-17T03:15:50.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.200+0000 7fcde7a63640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcde0074750 0x7fcde0156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:50.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.200+0000 7fcde7a63640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcde0074b60 0x7fcde015a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:50.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.200+0000 7fcde7a63640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcde0162f40 0x7fcde015aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:50.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.200+0000 7fcde7a63640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcde015f2b0 con 0x7fcde0074750 2024-09-17T03:15:50.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.200+0000 7fcde7a63640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcde00cea90 con 0x7fcde0074b60 2024-09-17T03:15:50.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.200+0000 7fcde7a63640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcde015f430 con 0x7fcde0162f40 2024-09-17T03:15:50.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.200+0000 7fcde57ff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcde0074750 0x7fcde0156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:50.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.200+0000 7fcde57ff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcde0074750 0x7fcde0156a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42316/0 (socket says 172.21.15.87:42316) 2024-09-17T03:15:50.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.200+0000 7fcde57ff640 1 -- 172.21.15.87:0/1312845055 learned_addr learned my addr 172.21.15.87:0/1312845055 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:50.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.201+0000 7fcde57ff640 1 -- 172.21.15.87:0/1312845055 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcde0162f40 msgr2=0x7fcde015aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:50.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.201+0000 7fcde6000640 1 --2- 172.21.15.87:0/1312845055 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcde0162f40 0x7fcde015aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:50.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.201+0000 7fcde4ffe640 1 --2- 172.21.15.87:0/1312845055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcde0074b60 0x7fcde015a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:50.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.201+0000 7fcde57ff640 1 --2- 172.21.15.87:0/1312845055 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcde0162f40 0x7fcde015aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:50.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.201+0000 7fcde57ff640 1 -- 172.21.15.87:0/1312845055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcde0074b60 msgr2=0x7fcde015a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:50.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.201+0000 7fcde57ff640 1 --2- 172.21.15.87:0/1312845055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcde0074b60 0x7fcde015a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:50.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.201+0000 7fcde57ff640 1 -- 172.21.15.87:0/1312845055 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde00b6ee0 con 0x7fcde0074750 2024-09-17T03:15:50.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.201+0000 7fcde4ffe640 1 --2- 172.21.15.87:0/1312845055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcde0074b60 0x7fcde015a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:15:50.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.201+0000 7fcde6000640 1 --2- 172.21.15.87:0/1312845055 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcde0162f40 0x7fcde015aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:15:50.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.201+0000 7fcde57ff640 1 --2- 172.21.15.87:0/1312845055 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcde0074750 0x7fcde0156a30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00d770 tx=0x7fcddc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:50.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.201+0000 7fcdd5ffb640 1 -- 172.21.15.87:0/1312845055 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc017070 con 0x7fcde0074750 2024-09-17T03:15:50.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.201+0000 7fcdd5ffb640 1 -- 172.21.15.87:0/1312845055 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcddc009d20 con 0x7fcde0074750 2024-09-17T03:15:50.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.201+0000 7fcde7a63640 1 -- 172.21.15.87:0/1312845055 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde015c130 con 0x7fcde0074750 2024-09-17T03:15:50.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.201+0000 7fcdd5ffb640 1 -- 172.21.15.87:0/1312845055 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcddc0106c0 con 0x7fcde0074750 2024-09-17T03:15:50.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.201+0000 7fcde7a63640 1 -- 172.21.15.87:0/1312845055 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde00b5530 con 0x7fcde0074750 2024-09-17T03:15:50.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.202+0000 7fcde7a63640 1 -- 172.21.15.87:0/1312845055 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcde0073e60 con 0x7fcde0074750 2024-09-17T03:15:50.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.203+0000 7fcdd5ffb640 1 -- 172.21.15.87:0/1312845055 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcddc01a070 con 0x7fcde0074750 2024-09-17T03:15:50.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.203+0000 7fcdd5ffb640 1 --2- 172.21.15.87:0/1312845055 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcdc4078bc0 0x7fcdc407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:50.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.203+0000 7fcdd5ffb640 1 -- 172.21.15.87:0/1312845055 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fcddc09ae90 con 0x7fcde0074750 2024-09-17T03:15:50.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.204+0000 7fcde4ffe640 1 --2- 172.21.15.87:0/1312845055 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcdc4078bc0 0x7fcdc407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:50.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.204+0000 7fcde4ffe640 1 --2- 172.21.15.87:0/1312845055 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcdc4078bc0 0x7fcdc407b0d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0033000 tx=0x7fcdd0004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:50.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.208+0000 7fcdd5ffb640 1 -- 172.21.15.87:0/1312845055 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fcddc0632e0 con 0x7fcde0074750 2024-09-17T03:15:50.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.452+0000 7fcde7a63640 1 -- 172.21.15.87:0/1312845055 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=cephfs_data"]} v 0) -- 0x7fcde00b4110 con 0x7fcde0074750 2024-09-17T03:15:50.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.455+0000 7fcdd5ffb640 1 -- 172.21.15.87:0/1312845055 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow r pool=cephfs_data"]}]=0 updated caps for client.0 v18) ==== 183+0+140 (secure 0 0 0) 0x7fcddc062be0 con 0x7fcde0074750 2024-09-17T03:15:50.454 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-09-17T03:15:50.454 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDH8+hmLskNLhAAd2M5ynGP8W6Pim11F18wYA== 2024-09-17T03:15:50.454 INFO:teuthology.orchestra.run.smithi087.stdout: caps mds = "allow" 2024-09-17T03:15:50.454 INFO:teuthology.orchestra.run.smithi087.stdout: caps mon = "allow r" 2024-09-17T03:15:50.454 INFO:teuthology.orchestra.run.smithi087.stdout: caps osd = "allow r pool=cephfs_data" 2024-09-17T03:15:50.454 INFO:teuthology.orchestra.run.smithi087.stderr:updated caps for client.0 2024-09-17T03:15:50.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.457+0000 7fcde7a63640 1 -- 172.21.15.87:0/1312845055 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcdc4078bc0 msgr2=0x7fcdc407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:50.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.457+0000 7fcde7a63640 1 --2- 172.21.15.87:0/1312845055 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcdc4078bc0 0x7fcdc407b0d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0033000 tx=0x7fcdd0004730 comp rx=0 tx=0).stop 2024-09-17T03:15:50.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.457+0000 7fcde7a63640 1 -- 172.21.15.87:0/1312845055 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcde0074750 msgr2=0x7fcde0156a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:50.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.457+0000 7fcde7a63640 1 --2- 172.21.15.87:0/1312845055 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcde0074750 0x7fcde0156a30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00d770 tx=0x7fcddc00dc30 comp rx=0 tx=0).stop 2024-09-17T03:15:50.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.457+0000 7fcde7a63640 1 -- 172.21.15.87:0/1312845055 shutdown_connections 2024-09-17T03:15:50.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.457+0000 7fcde7a63640 1 --2- 172.21.15.87:0/1312845055 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcdc4078bc0 0x7fcdc407b0d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:50.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.457+0000 7fcde7a63640 1 --2- 172.21.15.87:0/1312845055 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcde0162f40 0x7fcde015aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:50.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.457+0000 7fcde7a63640 1 --2- 172.21.15.87:0/1312845055 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcde0074b60 0x7fcde015a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:50.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.457+0000 7fcde7a63640 1 --2- 172.21.15.87:0/1312845055 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcde0074750 0x7fcde0156a30 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:50.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.457+0000 7fcde7a63640 1 -- 172.21.15.87:0/1312845055 >> 172.21.15.87:0/1312845055 conn(0x7fcde00cdc80 msgr2=0x7fcde00b1770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:50.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.457+0000 7fcde7a63640 1 -- 172.21.15.87:0/1312845055 shutdown_connections 2024-09-17T03:15:50.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.457+0000 7fcde7a63640 1 -- 172.21.15.87:0/1312845055 wait complete. 2024-09-17T03:15:50.468 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:15:50.468 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:15:50.484 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:15:50.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.639+0000 7f6a7a01d640 1 -- 172.21.15.87:0/1973613833 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6a740b7f10 msgr2=0x7f6a7415c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:50.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.639+0000 7f6a7a01d640 1 --2- 172.21.15.87:0/1973613833 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6a740b7f10 0x7f6a7415c5f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6a64005fd0 tx=0x7f6a6402a540 comp rx=0 tx=0).stop 2024-09-17T03:15:50.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.639+0000 7f6a7a01d640 1 -- 172.21.15.87:0/1973613833 shutdown_connections 2024-09-17T03:15:50.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.639+0000 7f6a7a01d640 1 --2- 172.21.15.87:0/1973613833 >> v2:172.21.15.149:3300/0 conn(0x7f6a740b3d60 0x7f6a740b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:50.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.639+0000 7f6a7a01d640 1 --2- 172.21.15.87:0/1973613833 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6a740b7f10 0x7f6a7415c5f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:50.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.639+0000 7f6a7a01d640 1 -- 172.21.15.87:0/1973613833 >> 172.21.15.87:0/1973613833 conn(0x7f6a74069330 msgr2=0x7f6a740ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:50.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.639+0000 7f6a7a01d640 1 -- 172.21.15.87:0/1973613833 shutdown_connections 2024-09-17T03:15:50.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.639+0000 7f6a7a01d640 1 -- 172.21.15.87:0/1973613833 wait complete. 2024-09-17T03:15:50.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.641+0000 7f6a7a01d640 1 Processor -- start 2024-09-17T03:15:50.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.641+0000 7f6a7a01d640 1 -- start start 2024-09-17T03:15:50.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.642+0000 7f6a7a01d640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6a740710e0 0x7f6a7415e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:50.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.642+0000 7f6a7a01d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a740b3d60 0x7f6a7406f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:50.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.642+0000 7f6a7a01d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a740b7f10 0x7f6a7406fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:50.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.642+0000 7f6a7a01d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6a740b7b10 con 0x7f6a740b7f10 2024-09-17T03:15:50.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.642+0000 7f6a7a01d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6a740d1c70 con 0x7f6a740b3d60 2024-09-17T03:15:50.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.642+0000 7f6a7a01d640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6a740b7c90 con 0x7f6a740710e0 2024-09-17T03:15:50.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.643+0000 7f6a73fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a740b7f10 0x7f6a7406fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:50.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.643+0000 7f6a73fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a740b7f10 0x7f6a7406fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42322/0 (socket says 172.21.15.87:42322) 2024-09-17T03:15:50.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.643+0000 7f6a73fff640 1 -- 172.21.15.87:0/3471377852 learned_addr learned my addr 172.21.15.87:0/3471377852 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:50.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.643+0000 7f6a72ffd640 1 --2- 172.21.15.87:0/3471377852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a740b3d60 0x7f6a7406f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:50.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.643+0000 7f6a737fe640 1 --2- 172.21.15.87:0/3471377852 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6a740710e0 0x7f6a7415e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:50.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.643+0000 7f6a73fff640 1 -- 172.21.15.87:0/3471377852 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6a740710e0 msgr2=0x7f6a7415e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:50.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.643+0000 7f6a73fff640 1 --2- 172.21.15.87:0/3471377852 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6a740710e0 0x7f6a7415e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:50.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.643+0000 7f6a73fff640 1 -- 172.21.15.87:0/3471377852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a740b3d60 msgr2=0x7f6a7406f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:50.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.643+0000 7f6a73fff640 1 --2- 172.21.15.87:0/3471377852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a740b3d60 0x7f6a7406f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:50.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.643+0000 7f6a73fff640 1 -- 172.21.15.87:0/3471377852 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a7406e020 con 0x7f6a740b7f10 2024-09-17T03:15:50.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.643+0000 7f6a737fe640 1 --2- 172.21.15.87:0/3471377852 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6a740710e0 0x7f6a7415e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:50.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.643+0000 7f6a72ffd640 1 --2- 172.21.15.87:0/3471377852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a740b3d60 0x7f6a7406f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:50.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.643+0000 7f6a73fff640 1 --2- 172.21.15.87:0/3471377852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a740b7f10 0x7f6a7406fa00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00c730 tx=0x7f6a6c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:50.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.643+0000 7f6a6bfff640 1 -- 172.21.15.87:0/3471377852 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c003b70 con 0x7f6a740b7f10 2024-09-17T03:15:50.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.643+0000 7f6a6bfff640 1 -- 172.21.15.87:0/3471377852 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a6c003d10 con 0x7f6a740b7f10 2024-09-17T03:15:50.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.643+0000 7f6a7a01d640 1 -- 172.21.15.87:0/3471377852 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a7406e300 con 0x7f6a740b7f10 2024-09-17T03:15:50.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.643+0000 7f6a6bfff640 1 -- 172.21.15.87:0/3471377852 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c00aab0 con 0x7f6a740b7f10 2024-09-17T03:15:50.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.643+0000 7f6a7a01d640 1 -- 172.21.15.87:0/3471377852 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a7406e8c0 con 0x7f6a740b7f10 2024-09-17T03:15:50.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.645+0000 7f6a7a01d640 1 -- 172.21.15.87:0/3471377852 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a74071e30 con 0x7f6a740b7f10 2024-09-17T03:15:50.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.645+0000 7f6a6bfff640 1 -- 172.21.15.87:0/3471377852 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6a6c00ac50 con 0x7f6a740b7f10 2024-09-17T03:15:50.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.646+0000 7f6a6bfff640 1 --2- 172.21.15.87:0/3471377852 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6a54078b90 0x7f6a5407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:50.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.646+0000 7f6a6bfff640 1 -- 172.21.15.87:0/3471377852 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f6a6c09ab60 con 0x7f6a740b7f10 2024-09-17T03:15:50.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.646+0000 7f6a737fe640 1 --2- 172.21.15.87:0/3471377852 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6a54078b90 0x7f6a5407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:50.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.647+0000 7f6a737fe640 1 --2- 172.21.15.87:0/3471377852 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6a54078b90 0x7f6a5407b0a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6a64033000 tx=0x7f6a64004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:50.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.650+0000 7f6a6bfff640 1 -- 172.21.15.87:0/3471377852 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6a740b46b0 con 0x7f6a740b7f10 2024-09-17T03:15:50.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.835+0000 7f6a7a01d640 1 -- 172.21.15.87:0/3471377852 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6a740b7100 con 0x7f6a740b7f10 2024-09-17T03:15:50.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.836+0000 7f6a6bfff640 1 -- 172.21.15.87:0/3471377852 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v50) ==== 80+0+0 (secure 0 0 0) 0x7f6a6c062fb0 con 0x7f6a740b7f10 2024-09-17T03:15:50.835 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-09-17T03:15:50.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.838+0000 7f6a7a01d640 1 -- 172.21.15.87:0/3471377852 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6a54078b90 msgr2=0x7f6a5407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:50.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.838+0000 7f6a7a01d640 1 --2- 172.21.15.87:0/3471377852 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6a54078b90 0x7f6a5407b0a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6a64033000 tx=0x7f6a64004730 comp rx=0 tx=0).stop 2024-09-17T03:15:50.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.838+0000 7f6a7a01d640 1 -- 172.21.15.87:0/3471377852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a740b7f10 msgr2=0x7f6a7406fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:50.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.838+0000 7f6a7a01d640 1 --2- 172.21.15.87:0/3471377852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a740b7f10 0x7f6a7406fa00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00c730 tx=0x7f6a6c00cbf0 comp rx=0 tx=0).stop 2024-09-17T03:15:50.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.838+0000 7f6a7a01d640 1 -- 172.21.15.87:0/3471377852 shutdown_connections 2024-09-17T03:15:50.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.838+0000 7f6a7a01d640 1 --2- 172.21.15.87:0/3471377852 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6a54078b90 0x7f6a5407b0a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:50.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.838+0000 7f6a7a01d640 1 --2- 172.21.15.87:0/3471377852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6a740b7f10 0x7f6a7406fa00 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:50.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.838+0000 7f6a7a01d640 1 --2- 172.21.15.87:0/3471377852 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6a740b3d60 0x7f6a7406f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:50.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.838+0000 7f6a7a01d640 1 --2- 172.21.15.87:0/3471377852 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6a740710e0 0x7f6a7415e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:50.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.838+0000 7f6a7a01d640 1 -- 172.21.15.87:0/3471377852 >> 172.21.15.87:0/3471377852 conn(0x7f6a74069330 msgr2=0x7f6a7415d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:50.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.838+0000 7f6a7a01d640 1 -- 172.21.15.87:0/3471377852 shutdown_connections 2024-09-17T03:15:50.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:50.838+0000 7f6a7a01d640 1 -- 172.21.15.87:0/3471377852 wait complete. 2024-09-17T03:15:50.848 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:15:50.848 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:15:50.864 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:15:51.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.018+0000 7ff6a9b50640 1 -- 172.21.15.87:0/1567304286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6a415b490 msgr2=0x7ff6a415c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:51.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.018+0000 7ff6a9b50640 1 --2- 172.21.15.87:0/1567304286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6a415b490 0x7ff6a415c220 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ff6a406ad50 tx=0x7ff69802a5f0 comp rx=0 tx=0).stop 2024-09-17T03:15:51.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.018+0000 7ff6a9b50640 1 -- 172.21.15.87:0/1567304286 shutdown_connections 2024-09-17T03:15:51.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.018+0000 7ff6a9b50640 1 --2- 172.21.15.87:0/1567304286 >> v2:172.21.15.149:3300/0 conn(0x7ff6a415c760 0x7ff6a40b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:51.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.018+0000 7ff6a9b50640 1 --2- 172.21.15.87:0/1567304286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6a415b490 0x7ff6a415c220 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:51.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.018+0000 7ff6a9b50640 1 -- 172.21.15.87:0/1567304286 >> 172.21.15.87:0/1567304286 conn(0x7ff6a40d1110 msgr2=0x7ff6a40d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:51.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.018+0000 7ff6a9b50640 1 -- 172.21.15.87:0/1567304286 shutdown_connections 2024-09-17T03:15:51.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.018+0000 7ff6a9b50640 1 -- 172.21.15.87:0/1567304286 wait complete. 2024-09-17T03:15:51.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.020+0000 7ff6a9b50640 1 Processor -- start 2024-09-17T03:15:51.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.020+0000 7ff6a9b50640 1 -- start start 2024-09-17T03:15:51.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.021+0000 7ff6a9b50640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6a40d04e0 0x7ff6a41f77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:51.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.021+0000 7ff6a9b50640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff6a415b490 0x7ff6a406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:51.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.021+0000 7ff6a9b50640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6a415c760 0x7ff6a41f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:51.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.021+0000 7ff6a9b50640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a40cc620 con 0x7ff6a415c760 2024-09-17T03:15:51.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.022+0000 7ff6a9b50640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a40cc4a0 con 0x7ff6a40d04e0 2024-09-17T03:15:51.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.022+0000 7ff6a9b50640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff6a40cc7a0 con 0x7ff6a415b490 2024-09-17T03:15:51.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.022+0000 7ff6a3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6a415c760 0x7ff6a41f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:51.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.022+0000 7ff6a3fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6a415c760 0x7ff6a41f2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42350/0 (socket says 172.21.15.87:42350) 2024-09-17T03:15:51.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.022+0000 7ff6a3fff640 1 -- 172.21.15.87:0/1527825639 learned_addr learned my addr 172.21.15.87:0/1527825639 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:51.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.022+0000 7ff6a2ffd640 1 --2- 172.21.15.87:0/1527825639 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff6a415b490 0x7ff6a406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:51.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.022+0000 7ff6a37fe640 1 --2- 172.21.15.87:0/1527825639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6a40d04e0 0x7ff6a41f77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:51.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.022+0000 7ff6a3fff640 1 -- 172.21.15.87:0/1527825639 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff6a415b490 msgr2=0x7ff6a406cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:51.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.022+0000 7ff6a3fff640 1 --2- 172.21.15.87:0/1527825639 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff6a415b490 0x7ff6a406cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:51.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.022+0000 7ff6a3fff640 1 -- 172.21.15.87:0/1527825639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6a40d04e0 msgr2=0x7ff6a41f77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:51.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.022+0000 7ff6a3fff640 1 --2- 172.21.15.87:0/1527825639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6a40d04e0 0x7ff6a41f77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:51.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.022+0000 7ff6a3fff640 1 -- 172.21.15.87:0/1527825639 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a409ad50 con 0x7ff6a415c760 2024-09-17T03:15:51.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.022+0000 7ff6a37fe640 1 --2- 172.21.15.87:0/1527825639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6a40d04e0 0x7ff6a41f77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:51.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.022+0000 7ff6a3fff640 1 --2- 172.21.15.87:0/1527825639 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6a415c760 0x7ff6a41f2de0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00ea10 tx=0x7ff69c00eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:51.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.022+0000 7ff68bfff640 1 -- 172.21.15.87:0/1527825639 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c00cde0 con 0x7ff6a415c760 2024-09-17T03:15:51.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.022+0000 7ff68bfff640 1 -- 172.21.15.87:0/1527825639 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff69c004c10 con 0x7ff6a415c760 2024-09-17T03:15:51.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.022+0000 7ff6a9b50640 1 -- 172.21.15.87:0/1527825639 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a414a1a0 con 0x7ff6a415c760 2024-09-17T03:15:51.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.022+0000 7ff68bfff640 1 -- 172.21.15.87:0/1527825639 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69c005020 con 0x7ff6a415c760 2024-09-17T03:15:51.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.022+0000 7ff6a9b50640 1 -- 172.21.15.87:0/1527825639 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a414a760 con 0x7ff6a415c760 2024-09-17T03:15:51.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.023+0000 7ff6a9b50640 1 -- 172.21.15.87:0/1527825639 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6a415e9e0 con 0x7ff6a415c760 2024-09-17T03:15:51.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.024+0000 7ff68bfff640 1 -- 172.21.15.87:0/1527825639 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff69c0051c0 con 0x7ff6a415c760 2024-09-17T03:15:51.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.025+0000 7ff68bfff640 1 --2- 172.21.15.87:0/1527825639 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff690078bc0 0x7ff69007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:51.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.025+0000 7ff68bfff640 1 -- 172.21.15.87:0/1527825639 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7ff69c09b910 con 0x7ff6a415c760 2024-09-17T03:15:51.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.025+0000 7ff6a37fe640 1 --2- 172.21.15.87:0/1527825639 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff690078bc0 0x7ff69007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:51.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.025+0000 7ff6a37fe640 1 --2- 172.21.15.87:0/1527825639 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff690078bc0 0x7ff69007b0d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff694000f60 tx=0x7ff694009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:51.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.029+0000 7ff68bfff640 1 -- 172.21.15.87:0/1527825639 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff6a415e9e0 con 0x7ff6a415c760 2024-09-17T03:15:51.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.213+0000 7ff6a9b50640 1 -- 172.21.15.87:0/1527825639 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff6a40746c0 con 0x7ff6a415c760 2024-09-17T03:15:51.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.214+0000 7ff68bfff640 1 -- 172.21.15.87:0/1527825639 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v50) ==== 80+0+0 (secure 0 0 0) 0x7ff69c063e70 con 0x7ff6a415c760 2024-09-17T03:15:51.213 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-09-17T03:15:51.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.216+0000 7ff6a9b50640 1 -- 172.21.15.87:0/1527825639 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff690078bc0 msgr2=0x7ff69007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:51.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.216+0000 7ff6a9b50640 1 --2- 172.21.15.87:0/1527825639 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff690078bc0 0x7ff69007b0d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff694000f60 tx=0x7ff694009040 comp rx=0 tx=0).stop 2024-09-17T03:15:51.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.216+0000 7ff6a9b50640 1 -- 172.21.15.87:0/1527825639 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6a415c760 msgr2=0x7ff6a41f2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:51.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.216+0000 7ff6a9b50640 1 --2- 172.21.15.87:0/1527825639 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6a415c760 0x7ff6a41f2de0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff69c00ea10 tx=0x7ff69c00eed0 comp rx=0 tx=0).stop 2024-09-17T03:15:51.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.216+0000 7ff6a9b50640 1 -- 172.21.15.87:0/1527825639 shutdown_connections 2024-09-17T03:15:51.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.216+0000 7ff6a9b50640 1 --2- 172.21.15.87:0/1527825639 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff690078bc0 0x7ff69007b0d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:51.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.216+0000 7ff6a9b50640 1 --2- 172.21.15.87:0/1527825639 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6a415c760 0x7ff6a41f2de0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:51.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.216+0000 7ff6a9b50640 1 --2- 172.21.15.87:0/1527825639 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff6a415b490 0x7ff6a406cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:51.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.216+0000 7ff6a9b50640 1 --2- 172.21.15.87:0/1527825639 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff6a40d04e0 0x7ff6a41f77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:51.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.216+0000 7ff6a9b50640 1 -- 172.21.15.87:0/1527825639 >> 172.21.15.87:0/1527825639 conn(0x7ff6a40d1110 msgr2=0x7ff6a415a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:51.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.216+0000 7ff6a9b50640 1 -- 172.21.15.87:0/1527825639 shutdown_connections 2024-09-17T03:15:51.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:51.216+0000 7ff6a9b50640 1 -- 172.21.15.87:0/1527825639 wait complete. 2024-09-17T03:15:51.226 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi087.front.sepia.ceph.com... 2024-09-17T03:15:51.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:51.226 DEBUG:teuthology.orchestra.run.smithi087:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:51.249 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:ceph-fuse[53555]: fuse finished with error 0 and tester_r 0 2024-09-17T03:15:51.251 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:15:57.352 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-09-17T03:15:57.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:57.353 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T03:15:57.373 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T03:15:57.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.540+0000 7f98ec539640 1 -- 172.21.15.87:0/2193793606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98e40b6600 msgr2=0x7f98e415f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:57.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.540+0000 7f98ec539640 1 --2- 172.21.15.87:0/2193793606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98e40b6600 0x7f98e415f100 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f98dc005fd0 tx=0x7f98dc02ab40 comp rx=0 tx=0).stop 2024-09-17T03:15:57.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.540+0000 7f98ec539640 1 -- 172.21.15.87:0/2193793606 shutdown_connections 2024-09-17T03:15:57.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.540+0000 7f98ec539640 1 --2- 172.21.15.87:0/2193793606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98e40b6600 0x7f98e415f100 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:57.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.540+0000 7f98ec539640 1 -- 172.21.15.87:0/2193793606 >> 172.21.15.87:0/2193793606 conn(0x7f98e40d4470 msgr2=0x7f98e40d4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:57.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.540+0000 7f98ec539640 1 -- 172.21.15.87:0/2193793606 shutdown_connections 2024-09-17T03:15:57.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.540+0000 7f98ec539640 1 -- 172.21.15.87:0/2193793606 wait complete. 2024-09-17T03:15:57.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.542+0000 7f98ec539640 1 Processor -- start 2024-09-17T03:15:57.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.542+0000 7f98ec539640 1 -- start start 2024-09-17T03:15:57.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.543+0000 7f98ec539640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98e415c2d0 0x7f98e41f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:57.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.543+0000 7f98ec539640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98e415ecf0 0x7f98e41f2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:57.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.543+0000 7f98ec539640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98e41f26c0 0x7f98e4159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:57.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.543+0000 7f98ec539640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f98e40d2fc0 con 0x7f98e41f26c0 2024-09-17T03:15:57.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.543+0000 7f98ec539640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f98e40d2e40 con 0x7f98e415c2d0 2024-09-17T03:15:57.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.543+0000 7f98ec539640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f98e40d3140 con 0x7f98e415ecf0 2024-09-17T03:15:57.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.544+0000 7f98eaad6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98e41f26c0 0x7f98e4159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:57.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.544+0000 7f98eaad6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98e41f26c0 0x7f98e4159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44798/0 (socket says 172.21.15.87:44798) 2024-09-17T03:15:57.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.544+0000 7f98eaad6640 1 -- 172.21.15.87:0/411737388 learned_addr learned my addr 172.21.15.87:0/411737388 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:57.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.544+0000 7f98ea2d5640 1 --2- 172.21.15.87:0/411737388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98e415c2d0 0x7f98e41f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:57.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.544+0000 7f98e9ad4640 1 --2- 172.21.15.87:0/411737388 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98e415ecf0 0x7f98e41f2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:57.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.544+0000 7f98eaad6640 1 -- 172.21.15.87:0/411737388 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98e415ecf0 msgr2=0x7f98e41f2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:57.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.544+0000 7f98eaad6640 1 --2- 172.21.15.87:0/411737388 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98e415ecf0 0x7f98e41f2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:57.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.544+0000 7f98eaad6640 1 -- 172.21.15.87:0/411737388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98e415c2d0 msgr2=0x7f98e41f5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:57.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.544+0000 7f98eaad6640 1 --2- 172.21.15.87:0/411737388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98e415c2d0 0x7f98e41f5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:57.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.544+0000 7f98eaad6640 1 -- 172.21.15.87:0/411737388 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e414bd60 con 0x7f98e41f26c0 2024-09-17T03:15:57.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.544+0000 7f98eaad6640 1 --2- 172.21.15.87:0/411737388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98e41f26c0 0x7f98e4159ed0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009bf0 tx=0x7f98dc02c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:57.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.544+0000 7f98daffd640 1 -- 172.21.15.87:0/411737388 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc044070 con 0x7f98e41f26c0 2024-09-17T03:15:57.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.544+0000 7f98daffd640 1 -- 172.21.15.87:0/411737388 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98dc004250 con 0x7f98e41f26c0 2024-09-17T03:15:57.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.544+0000 7f98ec539640 1 -- 172.21.15.87:0/411737388 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e414c040 con 0x7f98e41f26c0 2024-09-17T03:15:57.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.544+0000 7f98daffd640 1 -- 172.21.15.87:0/411737388 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc004680 con 0x7f98e41f26c0 2024-09-17T03:15:57.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.544+0000 7f98ec539640 1 -- 172.21.15.87:0/411737388 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e414c600 con 0x7f98e41f26c0 2024-09-17T03:15:57.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.546+0000 7f98ec539640 1 -- 172.21.15.87:0/411737388 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98e40d35f0 con 0x7f98e41f26c0 2024-09-17T03:15:57.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.546+0000 7f98daffd640 1 -- 172.21.15.87:0/411737388 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f98dc036400 con 0x7f98e41f26c0 2024-09-17T03:15:57.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.547+0000 7f98daffd640 1 --2- 172.21.15.87:0/411737388 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f98c8078bc0 0x7f98c807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:57.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.547+0000 7f98daffd640 1 -- 172.21.15.87:0/411737388 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5738+0+0 (secure 0 0 0) 0x7f98dc0bb520 con 0x7f98e41f26c0 2024-09-17T03:15:57.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.547+0000 7f98ea2d5640 1 --2- 172.21.15.87:0/411737388 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f98c8078bc0 0x7f98c807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:57.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.548+0000 7f98ea2d5640 1 --2- 172.21.15.87:0/411737388 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f98c8078bc0 0x7f98c807b0d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f98e0000f60 tx=0x7f98e0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:57.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.551+0000 7f98daffd640 1 -- 172.21.15.87:0/411737388 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f98dc032030 con 0x7f98e41f26c0 2024-09-17T03:15:57.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.761+0000 7f98ec539640 1 -- 172.21.15.87:0/411737388 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f98e4161750 con 0x7f98e41f26c0 2024-09-17T03:15:57.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.761+0000 7f98daffd640 1 -- 172.21.15.87:0/411737388 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) ==== 53+0+73 (secure 0 0 0) 0x7f98dc004820 con 0x7f98e41f26c0 2024-09-17T03:15:57.760 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T03:15:57.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.763+0000 7f98ec539640 1 -- 172.21.15.87:0/411737388 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f98c8078bc0 msgr2=0x7f98c807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:57.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.763+0000 7f98ec539640 1 --2- 172.21.15.87:0/411737388 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f98c8078bc0 0x7f98c807b0d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f98e0000f60 tx=0x7f98e0009040 comp rx=0 tx=0).stop 2024-09-17T03:15:57.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.763+0000 7f98ec539640 1 -- 172.21.15.87:0/411737388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98e41f26c0 msgr2=0x7f98e4159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:57.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.763+0000 7f98ec539640 1 --2- 172.21.15.87:0/411737388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98e41f26c0 0x7f98e4159ed0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009bf0 tx=0x7f98dc02c8b0 comp rx=0 tx=0).stop 2024-09-17T03:15:57.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.763+0000 7f98ec539640 1 -- 172.21.15.87:0/411737388 shutdown_connections 2024-09-17T03:15:57.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.763+0000 7f98ec539640 1 --2- 172.21.15.87:0/411737388 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f98c8078bc0 0x7f98c807b0d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:57.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.763+0000 7f98ec539640 1 --2- 172.21.15.87:0/411737388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98e41f26c0 0x7f98e4159ed0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:57.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.763+0000 7f98ec539640 1 --2- 172.21.15.87:0/411737388 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f98e415ecf0 0x7f98e41f2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:57.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.763+0000 7f98ec539640 1 --2- 172.21.15.87:0/411737388 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f98e415c2d0 0x7f98e41f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:57.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.763+0000 7f98ec539640 1 -- 172.21.15.87:0/411737388 >> 172.21.15.87:0/411737388 conn(0x7f98e40d4470 msgr2=0x7f98e40d4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:57.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.763+0000 7f98ec539640 1 -- 172.21.15.87:0/411737388 shutdown_connections 2024-09-17T03:15:57.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:57.763+0000 7f98ec539640 1 -- 172.21.15.87:0/411737388 wait complete. 2024-09-17T03:15:57.773 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T03:15:57.773 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T03:15:57.773 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-09-17T03:15:57.773 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T03:15:57.773 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:57.773 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T03:15:57.774 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T03:15:57.774 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T03:15:57.774 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:15:57.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:57.775 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-09-17T03:15:57.790 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:15:57.791 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:15:57.791 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:15:57.791 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:57.791 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-09-17T03:15:57.791 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:57.791 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:15:57.791 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-09-17T03:15:57.791 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-09-17T03:15:57.791 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:15:57.791 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-09-17T03:15:57.791 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-09-17T03:15:57.792 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:15:57.792 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:15:57.792 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T03:15:57.792 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:15:57.792 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T03:15:57.792 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42674sec preferred_lft 42674sec 2024-09-17T03:15:57.792 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link noprefixroute 2024-09-17T03:15:57.792 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:57.792 INFO:teuthology.orchestra.run.smithi087.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:15:57.792 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 26:32:f7:da:05:b7 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:15:57.792 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T03:15:57.793 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:57.793 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c822:ceff:fef0:4a99/64 scope link 2024-09-17T03:15:57.793 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:57.793 INFO:teuthology.orchestra.run.smithi087.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:15:57.793 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether b2:cb:c1:94:14:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:15:57.793 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::b0cb:c1ff:fe94:145d/64 scope link 2024-09-17T03:15:57.793 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:57.793 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:15:57.793 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 26:32:f7:da:05:b7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:15:57.793 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::2432:f7ff:feda:5b7/64 scope link 2024-09-17T03:15:57.793 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:15:57.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:57.794 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T03:15:57.854 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-09-17T03:15:57.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:57.856 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-09-17T03:15:57.911 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-09-17T03:15:57.912 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T03:15:57.912 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T03:15:57.912 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T03:15:57.912 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T03:15:57.912 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:15:57.912 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-09-17T03:15:57.912 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T03:15:57.913 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T03:15:57.913 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T03:15:57.913 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-09-17T03:15:57.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:15:58.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:15:58.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:58.038 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-09-17T03:15:58.052 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T03:15:58.053 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T03:15:58.053 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:15:58.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:58.053 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-09-17T03:15:58.117 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T03:15:58.117 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:15:58.118 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:58.173 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:15:58.173 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:15:58.174 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:58.229 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-09-17T03:15:58.293 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T03:15:58.350 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-09-17T03:15:58.350 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-09-17T03:15:58.350 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-09-17T03:15:58.350 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-09-17T03:15:58.350 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-09-17T03:15:58.350 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-09-17T03:15:58.350 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-09-17T03:15:58.350 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-09-17T03:15:58.350 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-09-17T03:15:58.351 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-09-17T03:15:58.351 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-09-17T03:15:58.351 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-09-17T03:15:58.351 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T03:15:58.351 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-09-17T03:15:58.351 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-09-17T03:15:58.351 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-09-17T03:15:58.351 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-09-17T03:15:58.351 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-09-17T03:15:58.351 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-09-17T03:15:58.351 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T03:15:58.351 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T03:15:58.351 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T03:15:58.352 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T03:15:58.352 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-09-17T03:15:58.352 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-09-17T03:15:58.352 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T03:15:58.352 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T03:15:58.352 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T03:15:58.352 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T03:15:58.352 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-09-17T03:15:58.352 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:15:58.352 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:15:58.352 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:15:58.352 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:15:58.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:58.353 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-09-17T03:15:58.405 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T03:15:58.405 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-17T03:15:58.448 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-09-17T03:15:58.472 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T03:15:58.530 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-09-17T03:15:58.530 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-09-17T03:15:58.530 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-09-17T03:15:58.530 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-09-17T03:15:58.531 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-09-17T03:15:58.531 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-09-17T03:15:58.531 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-09-17T03:15:58.531 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-09-17T03:15:58.531 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-09-17T03:15:58.531 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-09-17T03:15:58.531 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-09-17T03:15:58.531 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-09-17T03:15:58.531 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T03:15:58.531 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-09-17T03:15:58.531 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-09-17T03:15:58.531 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-09-17T03:15:58.531 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-09-17T03:15:58.532 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-09-17T03:15:58.532 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-09-17T03:15:58.532 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T03:15:58.532 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T03:15:58.532 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T03:15:58.532 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T03:15:58.532 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-09-17T03:15:58.532 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-09-17T03:15:58.532 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T03:15:58.532 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T03:15:58.532 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T03:15:58.532 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T03:15:58.532 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-09-17T03:15:58.532 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:15:58.533 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:15:58.533 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:15:58.533 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:15:58.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:58.533 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-09-17T03:15:58.541 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:2024-09-17T03:15:58.542+0000 7f0d7b019480 -1 init, newargv = 0x7f0d5c004640 newargc=15 2024-09-17T03:15:58.541 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:ceph-fuse[54183]: starting ceph client 2024-09-17T03:15:58.546 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:ceph-fuse[54183]: starting fuse 2024-09-17T03:15:58.557 INFO:teuthology.orchestra.run.smithi087.stdout:42 2024-09-17T03:15:58.557 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-17T03:15:58.557 DEBUG:teuthology.orchestra.run.smithi087:> sudo stdin-killer -- python3 -c ' 2024-09-17T03:15:58.557 DEBUG:teuthology.orchestra.run.smithi087:> import glob 2024-09-17T03:15:58.557 DEBUG:teuthology.orchestra.run.smithi087:> import re 2024-09-17T03:15:58.557 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-09-17T03:15:58.557 DEBUG:teuthology.orchestra.run.smithi087:> import subprocess 2024-09-17T03:15:58.558 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:15:58.558 DEBUG:teuthology.orchestra.run.smithi087:> def _find_admin_socket(client_name): 2024-09-17T03:15:58.558 DEBUG:teuthology.orchestra.run.smithi087:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T03:15:58.558 DEBUG:teuthology.orchestra.run.smithi087:> files = glob.glob(asok_path) 2024-09-17T03:15:58.558 DEBUG:teuthology.orchestra.run.smithi087:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T03:15:58.558 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:15:58.558 DEBUG:teuthology.orchestra.run.smithi087:> # Given a non-glob path, it better be there 2024-09-17T03:15:58.558 DEBUG:teuthology.orchestra.run.smithi087:> if "*" not in asok_path: 2024-09-17T03:15:58.558 DEBUG:teuthology.orchestra.run.smithi087:> assert(len(files) == 1) 2024-09-17T03:15:58.558 DEBUG:teuthology.orchestra.run.smithi087:> return files[0] 2024-09-17T03:15:58.558 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:15:58.558 DEBUG:teuthology.orchestra.run.smithi087:> for f in files: 2024-09-17T03:15:58.558 DEBUG:teuthology.orchestra.run.smithi087:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T03:15:58.558 DEBUG:teuthology.orchestra.run.smithi087:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T03:15:58.558 DEBUG:teuthology.orchestra.run.smithi087:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T03:15:58.559 DEBUG:teuthology.orchestra.run.smithi087:> contents = proc_f.read() 2024-09-17T03:15:58.559 DEBUG:teuthology.orchestra.run.smithi087:> if mountpoint in contents: 2024-09-17T03:15:58.559 DEBUG:teuthology.orchestra.run.smithi087:> return f 2024-09-17T03:15:58.559 DEBUG:teuthology.orchestra.run.smithi087:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T03:15:58.559 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:15:58.559 DEBUG:teuthology.orchestra.run.smithi087:> print(_find_admin_socket("client.0")) 2024-09-17T03:15:58.559 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-09-17T03:15:58.674 INFO:teuthology.orchestra.run.smithi087.stdout:/var/run/ceph/ceph-client.0.54183.asok 2024-09-17T03:15:58.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:15:58.685 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.54183.asok 2024-09-17T03:15:58.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:58.685 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.54183.asok status 2024-09-17T03:15:58.826 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-17T03:15:58.827 INFO:teuthology.orchestra.run.smithi087.stdout: "metadata": { 2024-09-17T03:15:58.827 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T03:15:58.827 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T03:15:58.827 INFO:teuthology.orchestra.run.smithi087.stdout: "entity_id": "0", 2024-09-17T03:15:58.827 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": "smithi087", 2024-09-17T03:15:58.827 INFO:teuthology.orchestra.run.smithi087.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T03:15:58.827 INFO:teuthology.orchestra.run.smithi087.stdout: "pid": "54183", 2024-09-17T03:15:58.827 INFO:teuthology.orchestra.run.smithi087.stdout: "root": "/" 2024-09-17T03:15:58.827 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:58.827 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_count": 0, 2024-09-17T03:15:58.827 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_pinned_count": 0, 2024-09-17T03:15:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 5679, 2024-09-17T03:15:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: "inst": { 2024-09-17T03:15:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: "name": { 2024-09-17T03:15:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "client", 2024-09-17T03:15:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: "num": 5679 2024-09-17T03:15:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-09-17T03:15:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-09-17T03:15:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-09-17T03:15:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 2937205837 2024-09-17T03:15:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:15:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-09-17T03:15:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-09-17T03:15:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-09-17T03:15:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 2937205837 2024-09-17T03:15:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:15:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: "inst_str": "client.5679 192.168.0.1:0/2937205837", 2024-09-17T03:15:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: "addr_str": "192.168.0.1:0/2937205837", 2024-09-17T03:15:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: "inode_count": 1, 2024-09-17T03:15:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_epoch": 20, 2024-09-17T03:15:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch": 52, 2024-09-17T03:15:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch_barrier": 0, 2024-09-17T03:15:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: "blocklisted": false, 2024-09-17T03:15:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs" 2024-09-17T03:15:58.829 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-17T03:15:58.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:58.835 DEBUG:teuthology.orchestra.run.smithi087:> stat --file-system '--printf=%T 2024-09-17T03:15:58.836 DEBUG:teuthology.orchestra.run.smithi087:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:58.851 INFO:teuthology.orchestra.run.smithi087.stdout:fuseblk 2024-09-17T03:15:58.851 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:58.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:15:58.852 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:15:58.916 DEBUG:teuthology.orchestra.run.smithi087:> stdin-killer -- python3 -c ' 2024-09-17T03:15:58.916 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-09-17T03:15:58.916 DEBUG:teuthology.orchestra.run.smithi087:> import errno 2024-09-17T03:15:58.917 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:15:58.917 DEBUG:teuthology.orchestra.run.smithi087:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2024-09-17T03:15:58.917 DEBUG:teuthology.orchestra.run.smithi087:> try: 2024-09-17T03:15:58.917 DEBUG:teuthology.orchestra.run.smithi087:> if False: 2024-09-17T03:15:58.917 DEBUG:teuthology.orchestra.run.smithi087:> ret = os.read(fd, 1024) 2024-09-17T03:15:58.917 DEBUG:teuthology.orchestra.run.smithi087:> else: 2024-09-17T03:15:58.917 DEBUG:teuthology.orchestra.run.smithi087:> os.write(fd, b'"'"'content'"'"') 2024-09-17T03:15:58.917 DEBUG:teuthology.orchestra.run.smithi087:> except OSError as e: 2024-09-17T03:15:58.917 DEBUG:teuthology.orchestra.run.smithi087:> if e.errno != errno.EPERM: 2024-09-17T03:15:58.917 DEBUG:teuthology.orchestra.run.smithi087:> raise 2024-09-17T03:15:58.917 DEBUG:teuthology.orchestra.run.smithi087:> else: 2024-09-17T03:15:58.917 DEBUG:teuthology.orchestra.run.smithi087:> raise RuntimeError("client does not check permission of data pool") 2024-09-17T03:15:58.917 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-09-17T03:15:59.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:15:59.035 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow w pool=cephfs_data' 2024-09-17T03:15:59.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.196+0000 7f7969227640 1 -- 172.21.15.87:0/633958166 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f796406d9f0 msgr2=0x7f796415c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:59.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.196+0000 7f7969227640 1 --2- 172.21.15.87:0/633958166 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f796406d9f0 0x7f796415c6b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f795c005fd0 tx=0x7f795c02a4d0 comp rx=0 tx=0).stop 2024-09-17T03:15:59.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.196+0000 7f7969227640 1 -- 172.21.15.87:0/633958166 shutdown_connections 2024-09-17T03:15:59.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.196+0000 7f7969227640 1 --2- 172.21.15.87:0/633958166 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f796406d9f0 0x7f796415c6b0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:59.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.196+0000 7f7969227640 1 --2- 172.21.15.87:0/633958166 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f79640b6f70 0x7f79640747c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:59.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.196+0000 7f7969227640 1 -- 172.21.15.87:0/633958166 >> 172.21.15.87:0/633958166 conn(0x7f79640cdbc0 msgr2=0x7f79640cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:59.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.196+0000 7f7969227640 1 -- 172.21.15.87:0/633958166 shutdown_connections 2024-09-17T03:15:59.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.196+0000 7f7969227640 1 -- 172.21.15.87:0/633958166 wait complete. 2024-09-17T03:15:59.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.198+0000 7f7969227640 1 Processor -- start 2024-09-17T03:15:59.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.198+0000 7f7969227640 1 -- start start 2024-09-17T03:15:59.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.199+0000 7f7969227640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f796406d9f0 0x7f79640cf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:59.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.199+0000 7f7969227640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7964072410 0x7f79640d0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:59.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.199+0000 7f7969227640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7964153c30 0x7f79641f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:59.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.199+0000 7f7969227640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f796415d360 con 0x7f796406d9f0 2024-09-17T03:15:59.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.199+0000 7f7969227640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f796415d1e0 con 0x7f7964072410 2024-09-17T03:15:59.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.199+0000 7f7969227640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f796415d4e0 con 0x7f7964153c30 2024-09-17T03:15:59.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.199+0000 7f7962d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f796406d9f0 0x7f79640cf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:59.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.200+0000 7f7962d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f796406d9f0 0x7f79640cf4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44828/0 (socket says 172.21.15.87:44828) 2024-09-17T03:15:59.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.200+0000 7f7962d9d640 1 -- 172.21.15.87:0/1123127594 learned_addr learned my addr 172.21.15.87:0/1123127594 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:59.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.200+0000 7f796359e640 1 --2- 172.21.15.87:0/1123127594 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7964153c30 0x7f79641f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:59.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.200+0000 7f796259c640 1 --2- 172.21.15.87:0/1123127594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7964072410 0x7f79640d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:59.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.200+0000 7f7962d9d640 1 -- 172.21.15.87:0/1123127594 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7964153c30 msgr2=0x7f79641f62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:59.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.200+0000 7f7962d9d640 1 --2- 172.21.15.87:0/1123127594 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7964153c30 0x7f79641f62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:59.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.200+0000 7f7962d9d640 1 -- 172.21.15.87:0/1123127594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7964072410 msgr2=0x7f79640d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:59.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.200+0000 7f7962d9d640 1 --2- 172.21.15.87:0/1123127594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7964072410 0x7f79640d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:59.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.200+0000 7f7962d9d640 1 -- 172.21.15.87:0/1123127594 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79641f97f0 con 0x7f796406d9f0 2024-09-17T03:15:59.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.200+0000 7f796259c640 1 --2- 172.21.15.87:0/1123127594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7964072410 0x7f79640d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:59.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.200+0000 7f7962d9d640 1 --2- 172.21.15.87:0/1123127594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f796406d9f0 0x7f79640cf4e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f795400ba50 tx=0x7f795400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:59.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.200+0000 7f795b7fe640 1 -- 172.21.15.87:0/1123127594 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954019070 con 0x7f796406d9f0 2024-09-17T03:15:59.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.200+0000 7f795b7fe640 1 -- 172.21.15.87:0/1123127594 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f795400cd00 con 0x7f796406d9f0 2024-09-17T03:15:59.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.200+0000 7f795b7fe640 1 -- 172.21.15.87:0/1123127594 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79540146a0 con 0x7f796406d9f0 2024-09-17T03:15:59.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.200+0000 7f7969227640 1 -- 172.21.15.87:0/1123127594 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79641f9ad0 con 0x7f796406d9f0 2024-09-17T03:15:59.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.200+0000 7f7969227640 1 -- 172.21.15.87:0/1123127594 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79640d0ca0 con 0x7f796406d9f0 2024-09-17T03:15:59.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.201+0000 7f7969227640 1 -- 172.21.15.87:0/1123127594 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f796415c6b0 con 0x7f796406d9f0 2024-09-17T03:15:59.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.202+0000 7f795b7fe640 1 -- 172.21.15.87:0/1123127594 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f795401c070 con 0x7f796406d9f0 2024-09-17T03:15:59.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.202+0000 7f795b7fe640 1 --2- 172.21.15.87:0/1123127594 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7944078bc0 0x7f794407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:59.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.203+0000 7f795b7fe640 1 -- 172.21.15.87:0/1123127594 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5738+0+0 (secure 0 0 0) 0x7f795409aa80 con 0x7f796406d9f0 2024-09-17T03:15:59.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.203+0000 7f796259c640 1 --2- 172.21.15.87:0/1123127594 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7944078bc0 0x7f794407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:59.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.203+0000 7f796259c640 1 --2- 172.21.15.87:0/1123127594 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7944078bc0 0x7f794407b0d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f79640cf040 tx=0x7f794c008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:59.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.207+0000 7f795b7fe640 1 -- 172.21.15.87:0/1123127594 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7954062f30 con 0x7f796406d9f0 2024-09-17T03:15:59.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.453+0000 7f7969227640 1 -- 172.21.15.87:0/1123127594 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=cephfs_data"]} v 0) -- 0x7f79640d0f20 con 0x7f796406d9f0 2024-09-17T03:15:59.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.456+0000 7f795b7fe640 1 -- 172.21.15.87:0/1123127594 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow w pool=cephfs_data"]}]=0 updated caps for client.0 v19) ==== 183+0+140 (secure 0 0 0) 0x7f7954062830 con 0x7f796406d9f0 2024-09-17T03:15:59.455 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-09-17T03:15:59.455 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDH8+hmLskNLhAAd2M5ynGP8W6Pim11F18wYA== 2024-09-17T03:15:59.455 INFO:teuthology.orchestra.run.smithi087.stdout: caps mds = "allow" 2024-09-17T03:15:59.455 INFO:teuthology.orchestra.run.smithi087.stdout: caps mon = "allow r" 2024-09-17T03:15:59.455 INFO:teuthology.orchestra.run.smithi087.stdout: caps osd = "allow w pool=cephfs_data" 2024-09-17T03:15:59.455 INFO:teuthology.orchestra.run.smithi087.stderr:updated caps for client.0 2024-09-17T03:15:59.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.458+0000 7f7969227640 1 -- 172.21.15.87:0/1123127594 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7944078bc0 msgr2=0x7f794407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:59.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.458+0000 7f7969227640 1 --2- 172.21.15.87:0/1123127594 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7944078bc0 0x7f794407b0d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f79640cf040 tx=0x7f794c008040 comp rx=0 tx=0).stop 2024-09-17T03:15:59.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.458+0000 7f7969227640 1 -- 172.21.15.87:0/1123127594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f796406d9f0 msgr2=0x7f79640cf4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:59.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.458+0000 7f7969227640 1 --2- 172.21.15.87:0/1123127594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f796406d9f0 0x7f79640cf4e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f795400ba50 tx=0x7f795400bf10 comp rx=0 tx=0).stop 2024-09-17T03:15:59.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.458+0000 7f7969227640 1 -- 172.21.15.87:0/1123127594 shutdown_connections 2024-09-17T03:15:59.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.458+0000 7f7969227640 1 --2- 172.21.15.87:0/1123127594 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7944078bc0 0x7f794407b0d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:59.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.458+0000 7f7969227640 1 --2- 172.21.15.87:0/1123127594 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7964153c30 0x7f79641f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:59.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.458+0000 7f7969227640 1 --2- 172.21.15.87:0/1123127594 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7964072410 0x7f79640d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:59.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.458+0000 7f7969227640 1 --2- 172.21.15.87:0/1123127594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f796406d9f0 0x7f79640cf4e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:59.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.458+0000 7f7969227640 1 -- 172.21.15.87:0/1123127594 >> 172.21.15.87:0/1123127594 conn(0x7f79640cdbc0 msgr2=0x7f7964071040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:59.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.458+0000 7f7969227640 1 -- 172.21.15.87:0/1123127594 shutdown_connections 2024-09-17T03:15:59.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.458+0000 7f7969227640 1 -- 172.21.15.87:0/1123127594 wait complete. 2024-09-17T03:15:59.468 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:15:59.468 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:15:59.484 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:15:59.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.638+0000 7f21c7a27640 1 -- 172.21.15.87:0/3073638374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21c00b4470 msgr2=0x7f21c015c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:59.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.638+0000 7f21c7a27640 1 --2- 172.21.15.87:0/3073638374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21c00b4470 0x7f21c015c5f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f21b0005fd0 tx=0x7f21b002a4f0 comp rx=0 tx=0).stop 2024-09-17T03:15:59.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.638+0000 7f21c7a27640 1 -- 172.21.15.87:0/3073638374 shutdown_connections 2024-09-17T03:15:59.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.638+0000 7f21c7a27640 1 --2- 172.21.15.87:0/3073638374 >> v2:172.21.15.149:3300/0 conn(0x7f21c00d0230 0x7f21c00d0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:59.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.638+0000 7f21c7a27640 1 --2- 172.21.15.87:0/3073638374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21c00b4470 0x7f21c015c5f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:59.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.638+0000 7f21c7a27640 1 -- 172.21.15.87:0/3073638374 >> 172.21.15.87:0/3073638374 conn(0x7f21c0069330 msgr2=0x7f21c00ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:59.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.638+0000 7f21c7a27640 1 -- 172.21.15.87:0/3073638374 shutdown_connections 2024-09-17T03:15:59.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.638+0000 7f21c7a27640 1 -- 172.21.15.87:0/3073638374 wait complete. 2024-09-17T03:15:59.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.640+0000 7f21c7a27640 1 Processor -- start 2024-09-17T03:15:59.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.640+0000 7f21c7a27640 1 -- start start 2024-09-17T03:15:59.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.641+0000 7f21c7a27640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c00b7f10 0x7f21c00c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:59.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.641+0000 7f21c7a27640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f21c00d0230 0x7f21c015ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:59.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.641+0000 7f21c7a27640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21c01515c0 0x7f21c015ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:59.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.641+0000 7f21c7a27640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f21c00d0b80 con 0x7f21c01515c0 2024-09-17T03:15:59.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.641+0000 7f21c7a27640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f21c00b7d20 con 0x7f21c00b7f10 2024-09-17T03:15:59.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.641+0000 7f21c7a27640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f21c00d0d00 con 0x7f21c00d0230 2024-09-17T03:15:59.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.641+0000 7f21c5fc4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21c01515c0 0x7f21c015ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:59.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.641+0000 7f21c5fc4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21c01515c0 0x7f21c015ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44844/0 (socket says 172.21.15.87:44844) 2024-09-17T03:15:59.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.641+0000 7f21c5fc4640 1 -- 172.21.15.87:0/1353928472 learned_addr learned my addr 172.21.15.87:0/1353928472 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:15:59.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.641+0000 7f21c4fc2640 1 --2- 172.21.15.87:0/1353928472 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f21c00d0230 0x7f21c015ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:59.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.641+0000 7f21c57c3640 1 --2- 172.21.15.87:0/1353928472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c00b7f10 0x7f21c00c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:59.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.641+0000 7f21c5fc4640 1 -- 172.21.15.87:0/1353928472 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f21c00d0230 msgr2=0x7f21c015ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:59.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.641+0000 7f21c5fc4640 1 --2- 172.21.15.87:0/1353928472 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f21c00d0230 0x7f21c015ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:59.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.641+0000 7f21c5fc4640 1 -- 172.21.15.87:0/1353928472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c00b7f10 msgr2=0x7f21c00c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:59.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.641+0000 7f21c5fc4640 1 --2- 172.21.15.87:0/1353928472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c00b7f10 0x7f21c00c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:59.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.642+0000 7f21c5fc4640 1 -- 172.21.15.87:0/1353928472 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c01f3c90 con 0x7f21c01515c0 2024-09-17T03:15:59.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.642+0000 7f21c57c3640 1 --2- 172.21.15.87:0/1353928472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c00b7f10 0x7f21c00c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:15:59.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.642+0000 7f21c5fc4640 1 --2- 172.21.15.87:0/1353928472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21c01515c0 0x7f21c015ef70 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f21b800e990 tx=0x7f21b800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:59.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.642+0000 7f21b5ffb640 1 -- 172.21.15.87:0/1353928472 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b8004d20 con 0x7f21c01515c0 2024-09-17T03:15:59.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.642+0000 7f21b5ffb640 1 -- 172.21.15.87:0/1353928472 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21b800cb00 con 0x7f21c01515c0 2024-09-17T03:15:59.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.642+0000 7f21c7a27640 1 -- 172.21.15.87:0/1353928472 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c01542c0 con 0x7f21c01515c0 2024-09-17T03:15:59.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.642+0000 7f21b5ffb640 1 -- 172.21.15.87:0/1353928472 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b8005020 con 0x7f21c01515c0 2024-09-17T03:15:59.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.642+0000 7f21c7a27640 1 -- 172.21.15.87:0/1353928472 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c00c5ad0 con 0x7f21c01515c0 2024-09-17T03:15:59.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.643+0000 7f21c7a27640 1 -- 172.21.15.87:0/1353928472 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c00d11d0 con 0x7f21c01515c0 2024-09-17T03:15:59.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.644+0000 7f21b5ffb640 1 -- 172.21.15.87:0/1353928472 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f21b80051c0 con 0x7f21c01515c0 2024-09-17T03:15:59.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.644+0000 7f21b5ffb640 1 --2- 172.21.15.87:0/1353928472 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f21a4078b90 0x7f21a407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:15:59.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.644+0000 7f21b5ffb640 1 -- 172.21.15.87:0/1353928472 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5738+0+0 (secure 0 0 0) 0x7f21b809b590 con 0x7f21c01515c0 2024-09-17T03:15:59.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.645+0000 7f21c57c3640 1 --2- 172.21.15.87:0/1353928472 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f21a4078b90 0x7f21a407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:15:59.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.645+0000 7f21c57c3640 1 --2- 172.21.15.87:0/1353928472 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f21a4078b90 0x7f21a407b0a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f21bc006fd0 tx=0x7f21bc008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:15:59.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.649+0000 7f21b5ffb640 1 -- 172.21.15.87:0/1353928472 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f21b8016030 con 0x7f21c01515c0 2024-09-17T03:15:59.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.838+0000 7f21c7a27640 1 -- 172.21.15.87:0/1353928472 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f21c01e7620 con 0x7f21c01515c0 2024-09-17T03:15:59.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.839+0000 7f21b5ffb640 1 -- 172.21.15.87:0/1353928472 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v52) ==== 80+0+0 (secure 0 0 0) 0x7f21b8063360 con 0x7f21c01515c0 2024-09-17T03:15:59.838 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-09-17T03:15:59.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.841+0000 7f21c7a27640 1 -- 172.21.15.87:0/1353928472 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f21a4078b90 msgr2=0x7f21a407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:59.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.841+0000 7f21c7a27640 1 --2- 172.21.15.87:0/1353928472 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f21a4078b90 0x7f21a407b0a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f21bc006fd0 tx=0x7f21bc008040 comp rx=0 tx=0).stop 2024-09-17T03:15:59.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.841+0000 7f21c7a27640 1 -- 172.21.15.87:0/1353928472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21c01515c0 msgr2=0x7f21c015ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:15:59.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.841+0000 7f21c7a27640 1 --2- 172.21.15.87:0/1353928472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21c01515c0 0x7f21c015ef70 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f21b800e990 tx=0x7f21b800ee50 comp rx=0 tx=0).stop 2024-09-17T03:15:59.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.841+0000 7f21c7a27640 1 -- 172.21.15.87:0/1353928472 shutdown_connections 2024-09-17T03:15:59.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.841+0000 7f21c7a27640 1 --2- 172.21.15.87:0/1353928472 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f21a4078b90 0x7f21a407b0a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:59.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.841+0000 7f21c7a27640 1 --2- 172.21.15.87:0/1353928472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21c01515c0 0x7f21c015ef70 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:59.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.841+0000 7f21c7a27640 1 --2- 172.21.15.87:0/1353928472 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f21c00d0230 0x7f21c015ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:59.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.841+0000 7f21c7a27640 1 --2- 172.21.15.87:0/1353928472 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f21c00b7f10 0x7f21c00c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:15:59.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.841+0000 7f21c7a27640 1 -- 172.21.15.87:0/1353928472 >> 172.21.15.87:0/1353928472 conn(0x7f21c0069330 msgr2=0x7f21c00cc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:15:59.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.841+0000 7f21c7a27640 1 -- 172.21.15.87:0/1353928472 shutdown_connections 2024-09-17T03:15:59.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:15:59.841+0000 7f21c7a27640 1 -- 172.21.15.87:0/1353928472 wait complete. 2024-09-17T03:15:59.851 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:15:59.851 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:15:59.867 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:16:00.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.020+0000 7f4696827640 1 -- 172.21.15.87:0/2121192923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4690072120 msgr2=0x7f469015efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:00.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.020+0000 7f4696827640 1 --2- 172.21.15.87:0/2121192923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4690072120 0x7f469015efd0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f46900b0c10 tx=0x7f468002a560 comp rx=0 tx=0).stop 2024-09-17T03:16:00.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.020+0000 7f4696827640 1 -- 172.21.15.87:0/2121192923 shutdown_connections 2024-09-17T03:16:00.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.020+0000 7f4696827640 1 --2- 172.21.15.87:0/2121192923 >> v2:172.21.15.149:3300/0 conn(0x7f46900b4110 0x7f46900b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:00.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.020+0000 7f4696827640 1 --2- 172.21.15.87:0/2121192923 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4690072530 0x7f469015f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:00.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.020+0000 7f4696827640 1 --2- 172.21.15.87:0/2121192923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4690072120 0x7f469015efd0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:00.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.020+0000 7f4696827640 1 -- 172.21.15.87:0/2121192923 >> 172.21.15.87:0/2121192923 conn(0x7f46900d1180 msgr2=0x7f46900d1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:00.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.020+0000 7f4696827640 1 -- 172.21.15.87:0/2121192923 shutdown_connections 2024-09-17T03:16:00.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.020+0000 7f4696827640 1 -- 172.21.15.87:0/2121192923 wait complete. 2024-09-17T03:16:00.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.022+0000 7f4696827640 1 Processor -- start 2024-09-17T03:16:00.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.022+0000 7f4696827640 1 -- start start 2024-09-17T03:16:00.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.023+0000 7f4696827640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4690072120 0x7f469014ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:00.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.023+0000 7f4696827640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4690072530 0x7f469006cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:00.023 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.023+0000 7f4696827640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46900b4110 0x7f469006d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:00.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.023+0000 7f4696827640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f46900745a0 con 0x7f46900b4110 2024-09-17T03:16:00.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.023+0000 7f4696827640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4690074420 con 0x7f4690072530 2024-09-17T03:16:00.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.023+0000 7f4696827640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4690074720 con 0x7f4690072120 2024-09-17T03:16:00.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.023+0000 7f4694dc4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46900b4110 0x7f469006d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:00.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.023+0000 7f4694dc4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46900b4110 0x7f469006d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44864/0 (socket says 172.21.15.87:44864) 2024-09-17T03:16:00.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.023+0000 7f4694dc4640 1 -- 172.21.15.87:0/950830162 learned_addr learned my addr 172.21.15.87:0/950830162 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:00.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.024+0000 7f468ffff640 1 --2- 172.21.15.87:0/950830162 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4690072120 0x7f469014ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:00.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.024+0000 7f468f7fe640 1 --2- 172.21.15.87:0/950830162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4690072530 0x7f469006cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:00.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.024+0000 7f4694dc4640 1 -- 172.21.15.87:0/950830162 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4690072120 msgr2=0x7f469014ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:00.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.024+0000 7f4694dc4640 1 --2- 172.21.15.87:0/950830162 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4690072120 0x7f469014ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:00.024 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.024+0000 7f4694dc4640 1 -- 172.21.15.87:0/950830162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4690072530 msgr2=0x7f469006cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:00.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.024+0000 7f4694dc4640 1 --2- 172.21.15.87:0/950830162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4690072530 0x7f469006cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:00.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.024+0000 7f4694dc4640 1 -- 172.21.15.87:0/950830162 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46901f6e80 con 0x7f46900b4110 2024-09-17T03:16:00.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.024+0000 7f4694dc4640 1 --2- 172.21.15.87:0/950830162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46900b4110 0x7f469006d320 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f468800ca80 tx=0x7f468800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:00.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.024+0000 7f468cff9640 1 -- 172.21.15.87:0/950830162 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4688007d90 con 0x7f46900b4110 2024-09-17T03:16:00.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.024+0000 7f468cff9640 1 -- 172.21.15.87:0/950830162 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4688004c30 con 0x7f46900b4110 2024-09-17T03:16:00.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.024+0000 7f468cff9640 1 -- 172.21.15.87:0/950830162 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4688005280 con 0x7f46900b4110 2024-09-17T03:16:00.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.024+0000 7f4696827640 1 -- 172.21.15.87:0/950830162 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46901f7140 con 0x7f46900b4110 2024-09-17T03:16:00.025 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.024+0000 7f4696827640 1 -- 172.21.15.87:0/950830162 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46901f7700 con 0x7f46900b4110 2024-09-17T03:16:00.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.025+0000 7f4696827640 1 -- 172.21.15.87:0/950830162 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4690151900 con 0x7f46900b4110 2024-09-17T03:16:00.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.026+0000 7f468cff9640 1 -- 172.21.15.87:0/950830162 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4688005420 con 0x7f46900b4110 2024-09-17T03:16:00.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.026+0000 7f468cff9640 1 --2- 172.21.15.87:0/950830162 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f467c078b90 0x7f467c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:00.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.027+0000 7f468cff9640 1 -- 172.21.15.87:0/950830162 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5738+0+0 (secure 0 0 0) 0x7f468809b650 con 0x7f46900b4110 2024-09-17T03:16:00.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.027+0000 7f468ffff640 1 --2- 172.21.15.87:0/950830162 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f467c078b90 0x7f467c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:00.026 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.027+0000 7f468ffff640 1 --2- 172.21.15.87:0/950830162 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f467c078b90 0x7f467c07b0a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4680033000 tx=0x7f4680004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:00.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.031+0000 7f468cff9640 1 -- 172.21.15.87:0/950830162 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4688014070 con 0x7f46900b4110 2024-09-17T03:16:00.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.213+0000 7f4696827640 1 -- 172.21.15.87:0/950830162 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f46900b09c0 con 0x7f46900b4110 2024-09-17T03:16:00.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.214+0000 7f468cff9640 1 -- 172.21.15.87:0/950830162 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v52) ==== 80+0+0 (secure 0 0 0) 0x7f46880634b0 con 0x7f46900b4110 2024-09-17T03:16:00.213 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-09-17T03:16:00.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.216+0000 7f4696827640 1 -- 172.21.15.87:0/950830162 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f467c078b90 msgr2=0x7f467c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:00.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.216+0000 7f4696827640 1 --2- 172.21.15.87:0/950830162 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f467c078b90 0x7f467c07b0a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4680033000 tx=0x7f4680004730 comp rx=0 tx=0).stop 2024-09-17T03:16:00.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.216+0000 7f4696827640 1 -- 172.21.15.87:0/950830162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46900b4110 msgr2=0x7f469006d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:00.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.216+0000 7f4696827640 1 --2- 172.21.15.87:0/950830162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46900b4110 0x7f469006d320 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f468800ca80 tx=0x7f468800cf40 comp rx=0 tx=0).stop 2024-09-17T03:16:00.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.216+0000 7f4696827640 1 -- 172.21.15.87:0/950830162 shutdown_connections 2024-09-17T03:16:00.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.216+0000 7f4696827640 1 --2- 172.21.15.87:0/950830162 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f467c078b90 0x7f467c07b0a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:00.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.216+0000 7f4696827640 1 --2- 172.21.15.87:0/950830162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46900b4110 0x7f469006d320 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:00.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.216+0000 7f4696827640 1 --2- 172.21.15.87:0/950830162 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4690072530 0x7f469006cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:00.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.216+0000 7f4696827640 1 --2- 172.21.15.87:0/950830162 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4690072120 0x7f469014ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:00.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.216+0000 7f4696827640 1 -- 172.21.15.87:0/950830162 >> 172.21.15.87:0/950830162 conn(0x7f46900d1180 msgr2=0x7f469015c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:00.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.216+0000 7f4696827640 1 -- 172.21.15.87:0/950830162 shutdown_connections 2024-09-17T03:16:00.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.216+0000 7f4696827640 1 -- 172.21.15.87:0/950830162 wait complete. 2024-09-17T03:16:00.226 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi087.front.sepia.ceph.com... 2024-09-17T03:16:00.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:16:00.226 DEBUG:teuthology.orchestra.run.smithi087:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T03:16:00.248 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:ceph-fuse[54183]: fuse finished with error 0 and tester_r 0 2024-09-17T03:16:00.250 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:16:00.319 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-09-17T03:16:00.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:16:00.319 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T03:16:00.334 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-17T03:16:00.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.488+0000 7fbfa7ef9640 1 -- 172.21.15.87:0/1250245634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf88003a30 msgr2=0x7fbf88003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:00.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.488+0000 7fbfa7ef9640 1 --2- 172.21.15.87:0/1250245634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf88003a30 0x7fbf88003ea0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fbf9000c640 tx=0x7fbf88005830 comp rx=0 tx=0).stop 2024-09-17T03:16:00.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.488+0000 7fbfa6496640 1 -- 172.21.15.87:0/1250245634 reap_dead start 2024-09-17T03:16:00.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.488+0000 7fbfa7ef9640 1 -- 172.21.15.87:0/1250245634 shutdown_connections 2024-09-17T03:16:00.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.488+0000 7fbfa7ef9640 1 -- 172.21.15.87:0/1250245634 >> 172.21.15.87:0/1250245634 conn(0x7fbfa00b3d10 msgr2=0x7fbfa00b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:00.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.488+0000 7fbfa7ef9640 1 -- 172.21.15.87:0/1250245634 shutdown_connections 2024-09-17T03:16:00.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.488+0000 7fbfa7ef9640 1 -- 172.21.15.87:0/1250245634 wait complete. 2024-09-17T03:16:00.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.490+0000 7fbfa7ef9640 1 Processor -- start 2024-09-17T03:16:00.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.490+0000 7fbfa7ef9640 1 -- start start 2024-09-17T03:16:00.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.491+0000 7fbfa7ef9640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf88003a30 0x7fbfa01d4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:00.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.491+0000 7fbfa7ef9640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbfa01f3c10 0x7fbfa01f4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:00.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.491+0000 7fbfa7ef9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfa01f71d0 0x7fbfa01f75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:00.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.491+0000 7fbfa7ef9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa00ce220 con 0x7fbfa01f71d0 2024-09-17T03:16:00.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.491+0000 7fbfa7ef9640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa00ce0a0 con 0x7fbf88003a30 2024-09-17T03:16:00.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.491+0000 7fbfa7ef9640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbfa00b1bf0 con 0x7fbfa01f3c10 2024-09-17T03:16:00.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.491+0000 7fbfa6496640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfa01f71d0 0x7fbfa01f75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:00.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.491+0000 7fbfa6496640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfa01f71d0 0x7fbfa01f75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44876/0 (socket says 172.21.15.87:44876) 2024-09-17T03:16:00.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.491+0000 7fbfa6496640 1 -- 172.21.15.87:0/1648029981 learned_addr learned my addr 172.21.15.87:0/1648029981 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:00.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.491+0000 7fbfa5494640 1 --2- 172.21.15.87:0/1648029981 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbfa01f3c10 0x7fbfa01f4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:00.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.491+0000 7fbfa5c95640 1 --2- 172.21.15.87:0/1648029981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf88003a30 0x7fbfa01d4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:00.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.491+0000 7fbfa6496640 1 -- 172.21.15.87:0/1648029981 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbfa01f3c10 msgr2=0x7fbfa01f4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:00.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.491+0000 7fbfa6496640 1 --2- 172.21.15.87:0/1648029981 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbfa01f3c10 0x7fbfa01f4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:00.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.491+0000 7fbfa6496640 1 -- 172.21.15.87:0/1648029981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf88003a30 msgr2=0x7fbfa01d4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:00.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.491+0000 7fbfa6496640 1 --2- 172.21.15.87:0/1648029981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf88003a30 0x7fbfa01d4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:00.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.491+0000 7fbfa6496640 1 -- 172.21.15.87:0/1648029981 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa009e0b0 con 0x7fbfa01f71d0 2024-09-17T03:16:00.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.492+0000 7fbfa6496640 1 --2- 172.21.15.87:0/1648029981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfa01f71d0 0x7fbfa01f75e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbfa00cc820 tx=0x7fbf98011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:00.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.492+0000 7fbf9e7fc640 1 -- 172.21.15.87:0/1648029981 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf98004230 con 0x7fbfa01f71d0 2024-09-17T03:16:00.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.492+0000 7fbf9e7fc640 1 -- 172.21.15.87:0/1648029981 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf98016070 con 0x7fbfa01f71d0 2024-09-17T03:16:00.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.492+0000 7fbfa7ef9640 1 -- 172.21.15.87:0/1648029981 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa01f7b20 con 0x7fbfa01f71d0 2024-09-17T03:16:00.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.492+0000 7fbf9e7fc640 1 -- 172.21.15.87:0/1648029981 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf9800fa90 con 0x7fbfa01f71d0 2024-09-17T03:16:00.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.492+0000 7fbfa7ef9640 1 -- 172.21.15.87:0/1648029981 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa01f4b40 con 0x7fbfa01f71d0 2024-09-17T03:16:00.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.493+0000 7fbfa7ef9640 1 -- 172.21.15.87:0/1648029981 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfa00cd990 con 0x7fbfa01f71d0 2024-09-17T03:16:00.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.493+0000 7fbf9e7fc640 1 -- 172.21.15.87:0/1648029981 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbf9800fc30 con 0x7fbfa01f71d0 2024-09-17T03:16:00.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.494+0000 7fbf9e7fc640 1 --2- 172.21.15.87:0/1648029981 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbf8c078b90 0x7fbf8c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:00.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.494+0000 7fbf9e7fc640 1 -- 172.21.15.87:0/1648029981 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5738+0+0 (secure 0 0 0) 0x7fbf9809b560 con 0x7fbfa01f71d0 2024-09-17T03:16:00.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.494+0000 7fbfa5c95640 1 --2- 172.21.15.87:0/1648029981 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbf8c078b90 0x7fbf8c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:00.494 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.495+0000 7fbfa5c95640 1 --2- 172.21.15.87:0/1648029981 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbf8c078b90 0x7fbf8c07b0a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbf90033000 tx=0x7fbf90001ea0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:00.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.498+0000 7fbf9e7fc640 1 -- 172.21.15.87:0/1648029981 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fbf980633c0 con 0x7fbfa01f71d0 2024-09-17T03:16:00.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.711+0000 7fbfa7ef9640 1 -- 172.21.15.87:0/1648029981 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fbfa00ce6d0 con 0x7fbfa01f71d0 2024-09-17T03:16:00.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.711+0000 7fbf9e7fc640 1 -- 172.21.15.87:0/1648029981 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) ==== 53+0+73 (secure 0 0 0) 0x7fbf98067df0 con 0x7fbfa01f71d0 2024-09-17T03:16:00.710 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-17T03:16:00.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.713+0000 7fbfa7ef9640 1 -- 172.21.15.87:0/1648029981 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbf8c078b90 msgr2=0x7fbf8c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:00.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.713+0000 7fbfa7ef9640 1 --2- 172.21.15.87:0/1648029981 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbf8c078b90 0x7fbf8c07b0a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fbf90033000 tx=0x7fbf90001ea0 comp rx=0 tx=0).stop 2024-09-17T03:16:00.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.713+0000 7fbfa7ef9640 1 -- 172.21.15.87:0/1648029981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfa01f71d0 msgr2=0x7fbfa01f75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:00.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.713+0000 7fbfa7ef9640 1 --2- 172.21.15.87:0/1648029981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfa01f71d0 0x7fbfa01f75e0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbfa00cc820 tx=0x7fbf98011b10 comp rx=0 tx=0).stop 2024-09-17T03:16:00.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.713+0000 7fbfa7ef9640 1 -- 172.21.15.87:0/1648029981 shutdown_connections 2024-09-17T03:16:00.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.713+0000 7fbfa7ef9640 1 --2- 172.21.15.87:0/1648029981 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbf8c078b90 0x7fbf8c07b0a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:00.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.713+0000 7fbfa7ef9640 1 --2- 172.21.15.87:0/1648029981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbfa01f71d0 0x7fbfa01f75e0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:00.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.713+0000 7fbfa7ef9640 1 --2- 172.21.15.87:0/1648029981 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbfa01f3c10 0x7fbfa01f4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:00.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.713+0000 7fbfa7ef9640 1 --2- 172.21.15.87:0/1648029981 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbf88003a30 0x7fbfa01d4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:00.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.713+0000 7fbfa7ef9640 1 -- 172.21.15.87:0/1648029981 >> 172.21.15.87:0/1648029981 conn(0x7fbfa00b3d10 msgr2=0x7fbfa01e6d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:00.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.713+0000 7fbfa7ef9640 1 -- 172.21.15.87:0/1648029981 shutdown_connections 2024-09-17T03:16:00.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00.713+0000 7fbfa7ef9640 1 -- 172.21.15.87:0/1648029981 wait complete. 2024-09-17T03:16:00.725 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-17T03:16:00.726 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-17T03:16:00.726 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-09-17T03:16:00.726 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-17T03:16:00.726 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-17T03:16:00.726 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-17T03:16:00.726 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-17T03:16:00.726 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-17T03:16:00.726 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:16:00.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:16:00.726 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-09-17T03:16:00.743 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-17T03:16:00.743 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-17T03:16:00.743 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-09-17T03:16:00.743 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:16:00.743 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-09-17T03:16:00.743 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:16:00.743 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:16:00.743 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-09-17T03:16:00.744 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-09-17T03:16:00.744 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:16:00.744 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-09-17T03:16:00.744 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-09-17T03:16:00.744 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-17T03:16:00.744 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:16:00.744 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-17T03:16:00.744 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:16:00.744 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-17T03:16:00.744 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42671sec preferred_lft 42671sec 2024-09-17T03:16:00.744 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link noprefixroute 2024-09-17T03:16:00.744 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:16:00.745 INFO:teuthology.orchestra.run.smithi087.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-17T03:16:00.745 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 26:32:f7:da:05:b7 brd ff:ff:ff:ff:ff:ff 2024-09-17T03:16:00.745 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-17T03:16:00.745 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:16:00.745 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c822:ceff:fef0:4a99/64 scope link 2024-09-17T03:16:00.745 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:16:00.745 INFO:teuthology.orchestra.run.smithi087.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:16:00.745 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether b2:cb:c1:94:14:5d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:16:00.746 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::b0cb:c1ff:fe94:145d/64 scope link 2024-09-17T03:16:00.746 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:16:00.746 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-17T03:16:00.746 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 26:32:f7:da:05:b7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:16:00.746 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::2432:f7ff:feda:5b7/64 scope link 2024-09-17T03:16:00.746 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-09-17T03:16:00.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:16:00.747 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-17T03:16:00.808 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-09-17T03:16:00.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:16:00.810 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-09-17T03:16:00.866 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-09-17T03:16:00.866 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-17T03:16:00.866 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-17T03:16:00.866 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-17T03:16:00.866 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-17T03:16:00.866 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-17T03:16:00.866 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-09-17T03:16:00.867 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-17T03:16:00.867 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-17T03:16:00.867 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-17T03:16:00.867 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-09-17T03:16:00.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-17T03:16:00.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:16:00.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:16:00.992 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-09-17T03:16:01.008 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T03:16:01.008 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T03:16:01.008 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:16:01.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:16:01.009 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-09-17T03:16:01.073 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-17T03:16:01.073 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:16:01.073 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-17T03:16:01.129 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-17T03:16:01.129 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-17T03:16:01.129 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:16:01.185 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-09-17T03:16:01.249 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T03:16:01.306 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-09-17T03:16:01.306 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-09-17T03:16:01.306 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-09-17T03:16:01.306 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-09-17T03:16:01.306 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-09-17T03:16:01.307 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-09-17T03:16:01.307 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-09-17T03:16:01.307 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-09-17T03:16:01.307 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-09-17T03:16:01.307 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-09-17T03:16:01.307 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-09-17T03:16:01.307 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-09-17T03:16:01.307 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T03:16:01.307 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-09-17T03:16:01.307 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-09-17T03:16:01.307 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-09-17T03:16:01.307 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-09-17T03:16:01.307 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-09-17T03:16:01.307 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-09-17T03:16:01.307 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T03:16:01.308 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T03:16:01.308 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T03:16:01.308 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T03:16:01.308 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-09-17T03:16:01.308 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-09-17T03:16:01.308 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T03:16:01.308 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T03:16:01.308 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T03:16:01.308 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T03:16:01.308 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-09-17T03:16:01.308 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:16:01.308 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:16:01.308 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:16:01.308 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:16:01.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:16:01.309 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-09-17T03:16:01.362 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-17T03:16:01.362 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-17T03:16:01.404 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-09-17T03:16:01.428 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-09-17T03:16:01.486 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-09-17T03:16:01.486 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-09-17T03:16:01.486 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-09-17T03:16:01.487 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-09-17T03:16:01.487 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-09-17T03:16:01.487 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-09-17T03:16:01.487 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-09-17T03:16:01.487 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-09-17T03:16:01.487 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-09-17T03:16:01.487 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-09-17T03:16:01.487 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-09-17T03:16:01.487 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/selinux 2024-09-17T03:16:01.487 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-09-17T03:16:01.487 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-09-17T03:16:01.487 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-09-17T03:16:01.487 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-09-17T03:16:01.488 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-09-17T03:16:01.488 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-09-17T03:16:01.488 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-09-17T03:16:01.488 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysctl.service 2024-09-17T03:16:01.488 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-17T03:16:01.488 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-17T03:16:01.488 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/nfs/rpc_pipefs 2024-09-17T03:16:01.488 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-09-17T03:16:01.488 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-09-17T03:16:01.488 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-0 2024-09-17T03:16:01.488 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-1 2024-09-17T03:16:01.488 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-2 2024-09-17T03:16:01.488 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-3 2024-09-17T03:16:01.489 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-09-17T03:16:01.489 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:16:01.489 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:16:01.489 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:16:01.489 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:16:01.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:16:01.490 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-09-17T03:16:01.496 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:2024-09-17T03:16:01.497+0000 7fec941cf480 -1 init, newargv = 0x7fec84001f00 newargc=15 2024-09-17T03:16:01.496 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:ceph-fuse[54810]: starting ceph client 2024-09-17T03:16:01.502 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:ceph-fuse[54810]: starting fuse 2024-09-17T03:16:01.511 INFO:teuthology.orchestra.run.smithi087.stdout:42 2024-09-17T03:16:01.511 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-09-17T03:16:01.511 DEBUG:teuthology.orchestra.run.smithi087:> sudo stdin-killer -- python3 -c ' 2024-09-17T03:16:01.512 DEBUG:teuthology.orchestra.run.smithi087:> import glob 2024-09-17T03:16:01.512 DEBUG:teuthology.orchestra.run.smithi087:> import re 2024-09-17T03:16:01.512 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-09-17T03:16:01.512 DEBUG:teuthology.orchestra.run.smithi087:> import subprocess 2024-09-17T03:16:01.512 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:16:01.512 DEBUG:teuthology.orchestra.run.smithi087:> def _find_admin_socket(client_name): 2024-09-17T03:16:01.512 DEBUG:teuthology.orchestra.run.smithi087:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-17T03:16:01.512 DEBUG:teuthology.orchestra.run.smithi087:> files = glob.glob(asok_path) 2024-09-17T03:16:01.512 DEBUG:teuthology.orchestra.run.smithi087:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-17T03:16:01.512 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:16:01.512 DEBUG:teuthology.orchestra.run.smithi087:> # Given a non-glob path, it better be there 2024-09-17T03:16:01.512 DEBUG:teuthology.orchestra.run.smithi087:> if "*" not in asok_path: 2024-09-17T03:16:01.513 DEBUG:teuthology.orchestra.run.smithi087:> assert(len(files) == 1) 2024-09-17T03:16:01.513 DEBUG:teuthology.orchestra.run.smithi087:> return files[0] 2024-09-17T03:16:01.513 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:16:01.513 DEBUG:teuthology.orchestra.run.smithi087:> for f in files: 2024-09-17T03:16:01.513 DEBUG:teuthology.orchestra.run.smithi087:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-17T03:16:01.513 DEBUG:teuthology.orchestra.run.smithi087:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-17T03:16:01.513 DEBUG:teuthology.orchestra.run.smithi087:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-17T03:16:01.513 DEBUG:teuthology.orchestra.run.smithi087:> contents = proc_f.read() 2024-09-17T03:16:01.513 DEBUG:teuthology.orchestra.run.smithi087:> if mountpoint in contents: 2024-09-17T03:16:01.513 DEBUG:teuthology.orchestra.run.smithi087:> return f 2024-09-17T03:16:01.513 DEBUG:teuthology.orchestra.run.smithi087:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-17T03:16:01.513 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:16:01.513 DEBUG:teuthology.orchestra.run.smithi087:> print(_find_admin_socket("client.0")) 2024-09-17T03:16:01.513 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-09-17T03:16:01.627 INFO:teuthology.orchestra.run.smithi087.stdout:/var/run/ceph/ceph-client.0.54810.asok 2024-09-17T03:16:01.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-17T03:16:01.637 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.54810.asok 2024-09-17T03:16:01.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:16:01.638 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.54810.asok status 2024-09-17T03:16:01.779 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-17T03:16:01.779 INFO:teuthology.orchestra.run.smithi087.stdout: "metadata": { 2024-09-17T03:16:01.779 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_sha1": "ce1b342bf8c6349a8c90cd607f83bff2443f21cf", 2024-09-17T03:16:01.779 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.3.0-4989-gce1b342b (ce1b342bf8c6349a8c90cd607f83bff2443f21cf) squid (dev)", 2024-09-17T03:16:01.779 INFO:teuthology.orchestra.run.smithi087.stdout: "entity_id": "0", 2024-09-17T03:16:01.779 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": "smithi087", 2024-09-17T03:16:01.779 INFO:teuthology.orchestra.run.smithi087.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-17T03:16:01.779 INFO:teuthology.orchestra.run.smithi087.stdout: "pid": "54810", 2024-09-17T03:16:01.779 INFO:teuthology.orchestra.run.smithi087.stdout: "root": "/" 2024-09-17T03:16:01.780 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:16:01.780 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_count": 0, 2024-09-17T03:16:01.780 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_pinned_count": 0, 2024-09-17T03:16:01.780 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 5709, 2024-09-17T03:16:01.780 INFO:teuthology.orchestra.run.smithi087.stdout: "inst": { 2024-09-17T03:16:01.780 INFO:teuthology.orchestra.run.smithi087.stdout: "name": { 2024-09-17T03:16:01.780 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "client", 2024-09-17T03:16:01.780 INFO:teuthology.orchestra.run.smithi087.stdout: "num": 5709 2024-09-17T03:16:01.780 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:16:01.780 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-09-17T03:16:01.780 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-09-17T03:16:01.780 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-09-17T03:16:01.780 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 299973093 2024-09-17T03:16:01.781 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-09-17T03:16:01.781 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:16:01.781 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-09-17T03:16:01.781 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-09-17T03:16:01.781 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-09-17T03:16:01.781 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 299973093 2024-09-17T03:16:01.781 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-09-17T03:16:01.781 INFO:teuthology.orchestra.run.smithi087.stdout: "inst_str": "client.5709 192.168.0.1:0/299973093", 2024-09-17T03:16:01.781 INFO:teuthology.orchestra.run.smithi087.stdout: "addr_str": "192.168.0.1:0/299973093", 2024-09-17T03:16:01.781 INFO:teuthology.orchestra.run.smithi087.stdout: "inode_count": 1, 2024-09-17T03:16:01.781 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_epoch": 20, 2024-09-17T03:16:01.781 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch": 52, 2024-09-17T03:16:01.781 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch_barrier": 0, 2024-09-17T03:16:01.781 INFO:teuthology.orchestra.run.smithi087.stdout: "blocklisted": false, 2024-09-17T03:16:01.782 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs" 2024-09-17T03:16:01.782 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-17T03:16:01.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:16:01.788 DEBUG:teuthology.orchestra.run.smithi087:> stat --file-system '--printf=%T 2024-09-17T03:16:01.788 DEBUG:teuthology.orchestra.run.smithi087:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-17T03:16:01.803 INFO:teuthology.orchestra.run.smithi087.stdout:fuseblk 2024-09-17T03:16:01.804 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-17T03:16:01.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:16:01.804 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-17T03:16:01.869 DEBUG:teuthology.orchestra.run.smithi087:> stdin-killer -- python3 -c ' 2024-09-17T03:16:01.869 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-09-17T03:16:01.869 DEBUG:teuthology.orchestra.run.smithi087:> import errno 2024-09-17T03:16:01.869 DEBUG:teuthology.orchestra.run.smithi087:> 2024-09-17T03:16:01.869 DEBUG:teuthology.orchestra.run.smithi087:> fd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR) 2024-09-17T03:16:01.870 DEBUG:teuthology.orchestra.run.smithi087:> try: 2024-09-17T03:16:01.870 DEBUG:teuthology.orchestra.run.smithi087:> if True: 2024-09-17T03:16:01.870 DEBUG:teuthology.orchestra.run.smithi087:> ret = os.read(fd, 1024) 2024-09-17T03:16:01.870 DEBUG:teuthology.orchestra.run.smithi087:> else: 2024-09-17T03:16:01.870 DEBUG:teuthology.orchestra.run.smithi087:> os.write(fd, b'"'"'content'"'"') 2024-09-17T03:16:01.870 DEBUG:teuthology.orchestra.run.smithi087:> except OSError as e: 2024-09-17T03:16:01.870 DEBUG:teuthology.orchestra.run.smithi087:> if e.errno != errno.EPERM: 2024-09-17T03:16:01.870 DEBUG:teuthology.orchestra.run.smithi087:> raise 2024-09-17T03:16:01.870 DEBUG:teuthology.orchestra.run.smithi087:> else: 2024-09-17T03:16:01.871 DEBUG:teuthology.orchestra.run.smithi087:> raise RuntimeError("client does not check permission of data pool") 2024-09-17T03:16:01.871 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-09-17T03:16:01.977 INFO:teuthology.orchestra.run.smithi087.stderr:Traceback (most recent call last): 2024-09-17T03:16:01.977 INFO:teuthology.orchestra.run.smithi087.stderr: File "", line 15, in 2024-09-17T03:16:01.977 INFO:teuthology.orchestra.run.smithi087.stderr:RuntimeError: client does not check permission of data pool 2024-09-17T03:16:01.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:01 stdin-killer INFO: command exited with status 1: exiting normally with same code! 2024-09-17T03:16:01.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:16:01.986 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mds allow mon 'allow r' osd 'allow rw pool=cephfs_data' 2024-09-17T03:16:02.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.147+0000 7f67f8172640 1 -- 172.21.15.87:0/3783466815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67f015c5d0 msgr2=0x7f67f0160c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:02.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.147+0000 7f67f8172640 1 --2- 172.21.15.87:0/3783466815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67f015c5d0 0x7f67f0160c50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f67e0005fd0 tx=0x7f67e002a5d0 comp rx=0 tx=0).stop 2024-09-17T03:16:02.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.147+0000 7f67f8172640 1 -- 172.21.15.87:0/3783466815 shutdown_connections 2024-09-17T03:16:02.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.147+0000 7f67f8172640 1 --2- 172.21.15.87:0/3783466815 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f67f00ccaf0 0x7f67f00ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:02.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.147+0000 7f67f8172640 1 --2- 172.21.15.87:0/3783466815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67f015c5d0 0x7f67f0160c50 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:02.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.147+0000 7f67f8172640 1 -- 172.21.15.87:0/3783466815 >> 172.21.15.87:0/3783466815 conn(0x7f67f0069120 msgr2=0x7f67f0069530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:02.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.147+0000 7f67f8172640 1 -- 172.21.15.87:0/3783466815 shutdown_connections 2024-09-17T03:16:02.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.147+0000 7f67f8172640 1 -- 172.21.15.87:0/3783466815 wait complete. 2024-09-17T03:16:02.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.149+0000 7f67f8172640 1 Processor -- start 2024-09-17T03:16:02.147 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.149+0000 7f67f8172640 1 -- start start 2024-09-17T03:16:02.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.150+0000 7f67f8172640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f67f00ccaf0 0x7f67f01f7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:02.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.150+0000 7f67f8172640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67f00d1fe0 0x7f67f00b0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:02.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.150+0000 7f67f8172640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67f015c5d0 0x7f67f015eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:02.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.150+0000 7f67f8172640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f67f00d0a90 con 0x7f67f015c5d0 2024-09-17T03:16:02.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.150+0000 7f67f8172640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f67f00d0910 con 0x7f67f00d1fe0 2024-09-17T03:16:02.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.150+0000 7f67f8172640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f67f00d0c10 con 0x7f67f00ccaf0 2024-09-17T03:16:02.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.150+0000 7f67f670f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67f015c5d0 0x7f67f015eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:02.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.150+0000 7f67f670f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67f015c5d0 0x7f67f015eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44890/0 (socket says 172.21.15.87:44890) 2024-09-17T03:16:02.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.150+0000 7f67f670f640 1 -- 172.21.15.87:0/394743286 learned_addr learned my addr 172.21.15.87:0/394743286 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:02.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.151+0000 7f67f5f0e640 1 --2- 172.21.15.87:0/394743286 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f67f00ccaf0 0x7f67f01f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:02.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.151+0000 7f67f570d640 1 --2- 172.21.15.87:0/394743286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67f00d1fe0 0x7f67f00b0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:02.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.151+0000 7f67f670f640 1 -- 172.21.15.87:0/394743286 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f67f00ccaf0 msgr2=0x7f67f01f7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:02.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.151+0000 7f67f670f640 1 --2- 172.21.15.87:0/394743286 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f67f00ccaf0 0x7f67f01f7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:02.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.151+0000 7f67f670f640 1 -- 172.21.15.87:0/394743286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67f00d1fe0 msgr2=0x7f67f00b0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:02.151 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.151+0000 7f67f670f640 1 --2- 172.21.15.87:0/394743286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67f00d1fe0 0x7f67f00b0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:02.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.151+0000 7f67f670f640 1 -- 172.21.15.87:0/394743286 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f01f3910 con 0x7f67f015c5d0 2024-09-17T03:16:02.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.151+0000 7f67f670f640 1 --2- 172.21.15.87:0/394743286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67f015c5d0 0x7f67f015eeb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f67e800cb00 tx=0x7f67e800a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:02.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.151+0000 7f67e67fc640 1 -- 172.21.15.87:0/394743286 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e800f450 con 0x7f67f015c5d0 2024-09-17T03:16:02.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.151+0000 7f67e67fc640 1 -- 172.21.15.87:0/394743286 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67e800f700 con 0x7f67f015c5d0 2024-09-17T03:16:02.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.151+0000 7f67f8172640 1 -- 172.21.15.87:0/394743286 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f01f3bf0 con 0x7f67f015c5d0 2024-09-17T03:16:02.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.151+0000 7f67e67fc640 1 -- 172.21.15.87:0/394743286 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67e800fb50 con 0x7f67f015c5d0 2024-09-17T03:16:02.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.151+0000 7f67f8172640 1 -- 172.21.15.87:0/394743286 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f01f44e0 con 0x7f67f015c5d0 2024-09-17T03:16:02.152 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.152+0000 7f67f8172640 1 -- 172.21.15.87:0/394743286 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67f01f3680 con 0x7f67f015c5d0 2024-09-17T03:16:02.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.153+0000 7f67e67fc640 1 -- 172.21.15.87:0/394743286 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f67e800fcf0 con 0x7f67f015c5d0 2024-09-17T03:16:02.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.153+0000 7f67e67fc640 1 --2- 172.21.15.87:0/394743286 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f67d4078b90 0x7f67d407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:02.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.153+0000 7f67e67fc640 1 -- 172.21.15.87:0/394743286 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5738+0+0 (secure 0 0 0) 0x7f67e809b920 con 0x7f67f015c5d0 2024-09-17T03:16:02.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.153+0000 7f67f5f0e640 1 --2- 172.21.15.87:0/394743286 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f67d4078b90 0x7f67d407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:02.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.154+0000 7f67f5f0e640 1 --2- 172.21.15.87:0/394743286 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f67d4078b90 0x7f67d407b0a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f67ec007fd0 tx=0x7f67ec009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.158+0000 7f67e67fc640 1 -- 172.21.15.87:0/394743286 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f67e8014030 con 0x7f67f015c5d0 2024-09-17T03:16:02.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.399+0000 7f67f8172640 1 -- 172.21.15.87:0/394743286 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]} v 0) -- 0x7f67f01f4110 con 0x7f67f015c5d0 2024-09-17T03:16:02.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.402+0000 7f67e67fc640 1 -- 172.21.15.87:0/394743286 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mds", "allow", "mon", "allow r", "osd", "allow rw pool=cephfs_data"]}]=0 updated caps for client.0 v20) ==== 184+0+141 (secure 0 0 0) 0x7f67f01f4110 con 0x7f67f015c5d0 2024-09-17T03:16:02.401 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-09-17T03:16:02.401 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQDH8+hmLskNLhAAd2M5ynGP8W6Pim11F18wYA== 2024-09-17T03:16:02.401 INFO:teuthology.orchestra.run.smithi087.stdout: caps mds = "allow" 2024-09-17T03:16:02.402 INFO:teuthology.orchestra.run.smithi087.stdout: caps mon = "allow r" 2024-09-17T03:16:02.402 INFO:teuthology.orchestra.run.smithi087.stdout: caps osd = "allow rw pool=cephfs_data" 2024-09-17T03:16:02.402 INFO:teuthology.orchestra.run.smithi087.stderr:updated caps for client.0 2024-09-17T03:16:02.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.404+0000 7f67f8172640 1 -- 172.21.15.87:0/394743286 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f67d4078b90 msgr2=0x7f67d407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:02.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.404+0000 7f67f8172640 1 --2- 172.21.15.87:0/394743286 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f67d4078b90 0x7f67d407b0a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f67ec007fd0 tx=0x7f67ec009040 comp rx=0 tx=0).stop 2024-09-17T03:16:02.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.404+0000 7f67f8172640 1 -- 172.21.15.87:0/394743286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67f015c5d0 msgr2=0x7f67f015eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:02.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.404+0000 7f67f8172640 1 --2- 172.21.15.87:0/394743286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67f015c5d0 0x7f67f015eeb0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f67e800cb00 tx=0x7f67e800a250 comp rx=0 tx=0).stop 2024-09-17T03:16:02.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.404+0000 7f67f8172640 1 -- 172.21.15.87:0/394743286 shutdown_connections 2024-09-17T03:16:02.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.404+0000 7f67f8172640 1 --2- 172.21.15.87:0/394743286 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f67d4078b90 0x7f67d407b0a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:02.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.404+0000 7f67f8172640 1 --2- 172.21.15.87:0/394743286 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67f015c5d0 0x7f67f015eeb0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:02.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.404+0000 7f67f8172640 1 --2- 172.21.15.87:0/394743286 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f67f00d1fe0 0x7f67f00b0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:02.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.404+0000 7f67f8172640 1 --2- 172.21.15.87:0/394743286 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f67f00ccaf0 0x7f67f01f7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:02.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.404+0000 7f67f8172640 1 -- 172.21.15.87:0/394743286 >> 172.21.15.87:0/394743286 conn(0x7f67f0069120 msgr2=0x7f67f00b70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:02.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.404+0000 7f67f8172640 1 -- 172.21.15.87:0/394743286 shutdown_connections 2024-09-17T03:16:02.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.404+0000 7f67f8172640 1 -- 172.21.15.87:0/394743286 wait complete. 2024-09-17T03:16:02.414 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-17T03:16:02.415 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:16:02.457 DEBUG:teuthology.orchestra.run.smithi149:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-17T03:16:02.501 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-17T03:16:02.501 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:16:02.502 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:16:02.502 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:16:02.518 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:16:02.518 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:16:02.576 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-17T03:16:02.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.730+0000 7f235260e640 1 -- 172.21.15.87:0/3019850428 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f234c0d0b40 msgr2=0x7f234c0d0f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:02.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.730+0000 7f235260e640 1 --2- 172.21.15.87:0/3019850428 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f234c0d0b40 0x7f234c0d0f50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2340005fd0 tx=0x7f234002a520 comp rx=0 tx=0).stop 2024-09-17T03:16:02.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.730+0000 7f235260e640 1 -- 172.21.15.87:0/3019850428 shutdown_connections 2024-09-17T03:16:02.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.730+0000 7f235260e640 1 --2- 172.21.15.87:0/3019850428 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f234c0ccf80 0x7f234c0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:02.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.730+0000 7f235260e640 1 --2- 172.21.15.87:0/3019850428 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f234c0d0b40 0x7f234c0d0f50 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:02.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.730+0000 7f235260e640 1 -- 172.21.15.87:0/3019850428 >> 172.21.15.87:0/3019850428 conn(0x7f234c0d2230 msgr2=0x7f234c0d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:02.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.730+0000 7f235260e640 1 -- 172.21.15.87:0/3019850428 shutdown_connections 2024-09-17T03:16:02.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.730+0000 7f235260e640 1 -- 172.21.15.87:0/3019850428 wait complete. 2024-09-17T03:16:02.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.732+0000 7f235260e640 1 Processor -- start 2024-09-17T03:16:02.731 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.732+0000 7f235260e640 1 -- start start 2024-09-17T03:16:02.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.733+0000 7f235260e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f234c0b76d0 0x7f234c1f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:02.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.733+0000 7f235260e640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f234c0ccf80 0x7f234c1f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:02.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.733+0000 7f235260e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f234c0d0b40 0x7f234c0b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:02.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.733+0000 7f235260e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f234c0b3040 con 0x7f234c0d0b40 2024-09-17T03:16:02.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.733+0000 7f235260e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f234c0b2ec0 con 0x7f234c0b76d0 2024-09-17T03:16:02.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.733+0000 7f235260e640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f234c0b19d0 con 0x7f234c0ccf80 2024-09-17T03:16:02.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.733+0000 7f2350bab640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f234c0d0b40 0x7f234c0b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:02.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.733+0000 7f2350bab640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f234c0d0b40 0x7f234c0b5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44914/0 (socket says 172.21.15.87:44914) 2024-09-17T03:16:02.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.734+0000 7f2350bab640 1 -- 172.21.15.87:0/158260467 learned_addr learned my addr 172.21.15.87:0/158260467 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:02.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.734+0000 7f234b7fe640 1 --2- 172.21.15.87:0/158260467 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f234c0ccf80 0x7f234c1f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:02.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.734+0000 7f2350bab640 1 -- 172.21.15.87:0/158260467 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f234c0ccf80 msgr2=0x7f234c1f76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:02.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.734+0000 7f2350bab640 1 --2- 172.21.15.87:0/158260467 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f234c0ccf80 0x7f234c1f76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:02.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.734+0000 7f2350bab640 1 -- 172.21.15.87:0/158260467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f234c0b76d0 msgr2=0x7f234c1f31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:02.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.734+0000 7f234bfff640 1 --2- 172.21.15.87:0/158260467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f234c0b76d0 0x7f234c1f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:02.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.734+0000 7f2350bab640 1 --2- 172.21.15.87:0/158260467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f234c0b76d0 0x7f234c1f31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:02.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.734+0000 7f2350bab640 1 -- 172.21.15.87:0/158260467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234c1e7ec0 con 0x7f234c0d0b40 2024-09-17T03:16:02.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.734+0000 7f234bfff640 1 --2- 172.21.15.87:0/158260467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f234c0b76d0 0x7f234c1f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:16:02.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.734+0000 7f2350bab640 1 --2- 172.21.15.87:0/158260467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f234c0d0b40 0x7f234c0b5bd0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f234400ca80 tx=0x7f234400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:02.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.734+0000 7f2348ff9640 1 -- 172.21.15.87:0/158260467 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234400adc0 con 0x7f234c0d0b40 2024-09-17T03:16:02.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.734+0000 7f2348ff9640 1 -- 172.21.15.87:0/158260467 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2344003a80 con 0x7f234c0d0b40 2024-09-17T03:16:02.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.734+0000 7f235260e640 1 -- 172.21.15.87:0/158260467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f234c1e81a0 con 0x7f234c0d0b40 2024-09-17T03:16:02.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.734+0000 7f2348ff9640 1 -- 172.21.15.87:0/158260467 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2344004030 con 0x7f234c0d0b40 2024-09-17T03:16:02.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.734+0000 7f235260e640 1 -- 172.21.15.87:0/158260467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f234c1e8760 con 0x7f234c0d0b40 2024-09-17T03:16:02.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.735+0000 7f235260e640 1 -- 172.21.15.87:0/158260467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f234c0cd390 con 0x7f234c0d0b40 2024-09-17T03:16:02.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.736+0000 7f2348ff9640 1 -- 172.21.15.87:0/158260467 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f23440041d0 con 0x7f234c0d0b40 2024-09-17T03:16:02.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.736+0000 7f2348ff9640 1 --2- 172.21.15.87:0/158260467 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f2338078bc0 0x7f233807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:02.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.737+0000 7f2348ff9640 1 -- 172.21.15.87:0/158260467 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5738+0+0 (secure 0 0 0) 0x7f234409b500 con 0x7f234c0d0b40 2024-09-17T03:16:02.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.737+0000 7f234bfff640 1 --2- 172.21.15.87:0/158260467 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f2338078bc0 0x7f233807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:02.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.737+0000 7f234bfff640 1 --2- 172.21.15.87:0/158260467 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f2338078bc0 0x7f233807b0d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f233c00a830 tx=0x7f233c008040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:02.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.741+0000 7f2348ff9640 1 -- 172.21.15.87:0/158260467 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f2344014070 con 0x7f234c0d0b40 2024-09-17T03:16:02.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.923+0000 7f235260e640 1 -- 172.21.15.87:0/158260467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f234c1f7c00 con 0x7f234c0d0b40 2024-09-17T03:16:02.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.923+0000 7f2348ff9640 1 -- 172.21.15.87:0/158260467 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v52) ==== 80+0+0 (secure 0 0 0) 0x7f234c1f7c00 con 0x7f234c0d0b40 2024-09-17T03:16:02.922 INFO:teuthology.orchestra.run.smithi087.stderr:listed 0 entries 2024-09-17T03:16:02.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.925+0000 7f235260e640 1 -- 172.21.15.87:0/158260467 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f2338078bc0 msgr2=0x7f233807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:02.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.925+0000 7f235260e640 1 --2- 172.21.15.87:0/158260467 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f2338078bc0 0x7f233807b0d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f233c00a830 tx=0x7f233c008040 comp rx=0 tx=0).stop 2024-09-17T03:16:02.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.925+0000 7f235260e640 1 -- 172.21.15.87:0/158260467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f234c0d0b40 msgr2=0x7f234c0b5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:02.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.925+0000 7f235260e640 1 --2- 172.21.15.87:0/158260467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f234c0d0b40 0x7f234c0b5bd0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f234400ca80 tx=0x7f234400cf40 comp rx=0 tx=0).stop 2024-09-17T03:16:02.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.925+0000 7f235260e640 1 -- 172.21.15.87:0/158260467 shutdown_connections 2024-09-17T03:16:02.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.925+0000 7f235260e640 1 --2- 172.21.15.87:0/158260467 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f2338078bc0 0x7f233807b0d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:02.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.925+0000 7f235260e640 1 --2- 172.21.15.87:0/158260467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f234c0d0b40 0x7f234c0b5bd0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:02.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.925+0000 7f235260e640 1 --2- 172.21.15.87:0/158260467 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f234c0ccf80 0x7f234c1f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:02.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.925+0000 7f235260e640 1 --2- 172.21.15.87:0/158260467 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f234c0b76d0 0x7f234c1f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:02.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.925+0000 7f235260e640 1 -- 172.21.15.87:0/158260467 >> 172.21.15.87:0/158260467 conn(0x7f234c0d2230 msgr2=0x7f234c0cdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:02.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.925+0000 7f235260e640 1 -- 172.21.15.87:0/158260467 shutdown_connections 2024-09-17T03:16:02.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:02.925+0000 7f235260e640 1 -- 172.21.15.87:0/158260467 wait complete. 2024-09-17T03:16:02.935 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi087.front.sepia.ceph.com... 2024-09-17T03:16:02.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:16:02.935 DEBUG:teuthology.orchestra.run.smithi087:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-17T03:16:02.958 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi087.stderr:ceph-fuse[54810]: fuse finished with error 0 and tester_r 0 2024-09-17T03:16:02.960 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-09-17T03:16:02.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:16:02.960 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-17T03:16:03.074 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:16:03.075 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:16:03.075 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:16:03.094 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=1994716,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=3199368k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20815 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=1599680k,nr_inodes=399920,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=3199368k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:16:03.095 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:16:03.095 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:16:03.095 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:16:03.110 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=156 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:16:03.111 INFO:tasks.cephfs.mount:Terminating background process 2024-09-17T03:16:03.111 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:16:03.111 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:16:03.153 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=1994716,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=3199368k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20815 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=1599680k,nr_inodes=399920,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=3199368k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:16:03.153 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:16:03.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.266+0000 7fb28cb7b640 1 -- 172.21.15.87:0/2721711313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb270003a50 msgr2=0x7fb270003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:03.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.266+0000 7fb28cb7b640 1 --2- 172.21.15.87:0/2721711313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb270003a50 0x7fb270003ec0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb2780016c0 tx=0x7fb270005830 comp rx=0 tx=0).stop 2024-09-17T03:16:03.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.266+0000 7fb28cb7b640 1 -- 172.21.15.87:0/2721711313 shutdown_connections 2024-09-17T03:16:03.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.266+0000 7fb28cb7b640 1 --2- 172.21.15.87:0/2721711313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb270003a50 0x7fb270003ec0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:03.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.266+0000 7fb28cb7b640 1 -- 172.21.15.87:0/2721711313 >> 172.21.15.87:0/2721711313 conn(0x7fb288069900 msgr2=0x7fb2880d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:03.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.266+0000 7fb28cb7b640 1 -- 172.21.15.87:0/2721711313 shutdown_connections 2024-09-17T03:16:03.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.266+0000 7fb28cb7b640 1 -- 172.21.15.87:0/2721711313 wait complete. 2024-09-17T03:16:03.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.268+0000 7fb28cb7b640 1 Processor -- start 2024-09-17T03:16:03.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.268+0000 7fb28cb7b640 1 -- start start 2024-09-17T03:16:03.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.269+0000 7fb28cb7b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb270003a50 0x7fb28814ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:03.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.269+0000 7fb28cb7b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb28814fc40 0x7fb2880cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:03.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.269+0000 7fb28cb7b640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb2880cdcd0 0x7fb2880cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:03.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.269+0000 7fb28cb7b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb28815f890 con 0x7fb28814fc40 2024-09-17T03:16:03.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.269+0000 7fb28cb7b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb2880cf750 con 0x7fb270003a50 2024-09-17T03:16:03.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.269+0000 7fb28cb7b640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb28815fa10 con 0x7fb2880cdcd0 2024-09-17T03:16:03.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.269+0000 7fb285d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb28814fc40 0x7fb2880cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:03.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.269+0000 7fb285d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb28814fc40 0x7fb2880cd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44942/0 (socket says 172.21.15.87:44942) 2024-09-17T03:16:03.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.269+0000 7fb285d9b640 1 -- 172.21.15.87:0/3287984622 learned_addr learned my addr 172.21.15.87:0/3287984622 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:03.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.269+0000 7fb285d9b640 1 -- 172.21.15.87:0/3287984622 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb2880cdcd0 msgr2=0x7fb2880cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:03.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.269+0000 7fb285d9b640 1 --2- 172.21.15.87:0/3287984622 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb2880cdcd0 0x7fb2880cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:03.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.269+0000 7fb28659c640 1 --2- 172.21.15.87:0/3287984622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb270003a50 0x7fb28814ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:03.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.269+0000 7fb285d9b640 1 -- 172.21.15.87:0/3287984622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb270003a50 msgr2=0x7fb28814ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:03.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.269+0000 7fb285d9b640 1 --2- 172.21.15.87:0/3287984622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb270003a50 0x7fb28814ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:03.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.269+0000 7fb285d9b640 1 -- 172.21.15.87:0/3287984622 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2880ce0e0 con 0x7fb28814fc40 2024-09-17T03:16:03.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.270+0000 7fb285d9b640 1 --2- 172.21.15.87:0/3287984622 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb28814fc40 0x7fb2880cd790 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb27c00e5a0 tx=0x7fb27c00ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:03.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.270+0000 7fb276ffd640 1 -- 172.21.15.87:0/3287984622 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c015020 con 0x7fb28814fc40 2024-09-17T03:16:03.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.270+0000 7fb276ffd640 1 -- 172.21.15.87:0/3287984622 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb27c010980 con 0x7fb28814fc40 2024-09-17T03:16:03.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.270+0000 7fb28cb7b640 1 -- 172.21.15.87:0/3287984622 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2880c95f0 con 0x7fb28814fc40 2024-09-17T03:16:03.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.270+0000 7fb276ffd640 1 -- 172.21.15.87:0/3287984622 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27c010dd0 con 0x7fb28814fc40 2024-09-17T03:16:03.270 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.270+0000 7fb28cb7b640 1 -- 172.21.15.87:0/3287984622 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2880c8830 con 0x7fb28814fc40 2024-09-17T03:16:03.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.271+0000 7fb28cb7b640 1 -- 172.21.15.87:0/3287984622 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2880d1d30 con 0x7fb28814fc40 2024-09-17T03:16:03.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.272+0000 7fb276ffd640 1 -- 172.21.15.87:0/3287984622 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb27c018020 con 0x7fb28814fc40 2024-09-17T03:16:03.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.272+0000 7fb276ffd640 1 --2- 172.21.15.87:0/3287984622 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb268078b90 0x7fb26807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:03.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.272+0000 7fb276ffd640 1 -- 172.21.15.87:0/3287984622 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5738+0+0 (secure 0 0 0) 0x7fb27c09b2e0 con 0x7fb28814fc40 2024-09-17T03:16:03.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.272+0000 7fb28659c640 1 --2- 172.21.15.87:0/3287984622 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb268078b90 0x7fb26807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:03.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.273+0000 7fb28659c640 1 --2- 172.21.15.87:0/3287984622 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb268078b90 0x7fb26807b0a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb278032000 tx=0x7fb278001190 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:03.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.277+0000 7fb276ffd640 1 -- 172.21.15.87:0/3287984622 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb2880d1d30 con 0x7fb28814fc40 2024-09-17T03:16:03.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.490+0000 7fb28cb7b640 1 -- 172.21.15.87:0/3287984622 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2880c82e0 con 0x7fb28814fc40 2024-09-17T03:16:03.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.491+0000 7fb276ffd640 1 -- 172.21.15.87:0/3287984622 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5229 (secure 0 0 0) 0x7fb27c01f020 con 0x7fb28814fc40 2024-09-17T03:16:03.490 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:16:03.490 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":20,"btime":"2024-09-17T03:15:41:361821+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5104,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/1606650259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1606650259},{"type":"v1","addr":"172.21.15.87:6834","nonce":1606650259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":5116,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6836/3876173555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3876173555},{"type":"v1","addr":"172.21.15.149:6836","nonce":3876173555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1657329632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":1657329632},{"type":"v1","addr":"172.21.15.87:6835","nonce":1657329632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:15:38.707286+0000","modified":"2024-09-17T03:15:40.712230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5487":{"gid":5487,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6837/3562056427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6835","nonce":3562056427},{"type":"v1","addr":"172.21.15.149:6837","nonce":3562056427}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5487,"qdb_cluster":[5487]},"id":3}]} 2024-09-17T03:16:03.490 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 20 2024-09-17T03:16:03.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.493+0000 7fb28cb7b640 1 -- 172.21.15.87:0/3287984622 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb268078b90 msgr2=0x7fb26807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:03.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.493+0000 7fb28cb7b640 1 --2- 172.21.15.87:0/3287984622 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb268078b90 0x7fb26807b0a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb278032000 tx=0x7fb278001190 comp rx=0 tx=0).stop 2024-09-17T03:16:03.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.493+0000 7fb28cb7b640 1 -- 172.21.15.87:0/3287984622 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb28814fc40 msgr2=0x7fb2880cd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:03.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.493+0000 7fb28cb7b640 1 --2- 172.21.15.87:0/3287984622 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb28814fc40 0x7fb2880cd790 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb27c00e5a0 tx=0x7fb27c00ea60 comp rx=0 tx=0).stop 2024-09-17T03:16:03.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.493+0000 7fb28cb7b640 1 -- 172.21.15.87:0/3287984622 shutdown_connections 2024-09-17T03:16:03.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.493+0000 7fb28cb7b640 1 --2- 172.21.15.87:0/3287984622 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb268078b90 0x7fb26807b0a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:03.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.493+0000 7fb28cb7b640 1 --2- 172.21.15.87:0/3287984622 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb2880cdcd0 0x7fb2880cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:03.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.493+0000 7fb28cb7b640 1 --2- 172.21.15.87:0/3287984622 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb28814fc40 0x7fb2880cd790 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:03.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.493+0000 7fb28cb7b640 1 --2- 172.21.15.87:0/3287984622 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb270003a50 0x7fb28814ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:03.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.493+0000 7fb28cb7b640 1 -- 172.21.15.87:0/3287984622 >> 172.21.15.87:0/3287984622 conn(0x7fb288069900 msgr2=0x7fb28806de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:03.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.493+0000 7fb28cb7b640 1 -- 172.21.15.87:0/3287984622 shutdown_connections 2024-09-17T03:16:03.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.493+0000 7fb28cb7b640 1 -- 172.21.15.87:0/3287984622 wait complete. 2024-09-17T03:16:03.504 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:16:03.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.618+0000 7f66b6c27640 1 -- 172.21.15.87:0/1791877863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66b0074fb0 msgr2=0x7f66b00c4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:03.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.618+0000 7f66b6c27640 1 --2- 172.21.15.87:0/1791877863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66b0074fb0 0x7f66b00c4f00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f66a8005fd0 tx=0x7f66a802a8a0 comp rx=0 tx=0).stop 2024-09-17T03:16:03.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.618+0000 7f66b6c27640 1 -- 172.21.15.87:0/1791877863 shutdown_connections 2024-09-17T03:16:03.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.618+0000 7f66b6c27640 1 --2- 172.21.15.87:0/1791877863 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66b0074fb0 0x7f66b00c4f00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:03.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.618+0000 7f66b6c27640 1 --2- 172.21.15.87:0/1791877863 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f66b00cc590 0x7f66b015c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:03.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.618+0000 7f66b6c27640 1 -- 172.21.15.87:0/1791877863 >> 172.21.15.87:0/1791877863 conn(0x7f66b006ee70 msgr2=0x7f66b006f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:03.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.618+0000 7f66b6c27640 1 -- 172.21.15.87:0/1791877863 shutdown_connections 2024-09-17T03:16:03.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.618+0000 7f66b6c27640 1 -- 172.21.15.87:0/1791877863 wait complete. 2024-09-17T03:16:03.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.620+0000 7f66b6c27640 1 Processor -- start 2024-09-17T03:16:03.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.620+0000 7f66b6c27640 1 -- start start 2024-09-17T03:16:03.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.621+0000 7f66b6c27640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66b00cc590 0x7f66b00b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:03.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.621+0000 7f66b6c27640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f66b015f0c0 0x7f66b01f33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:03.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.621+0000 7f66b6c27640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66b0156b90 0x7f66b00c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:03.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.621+0000 7f66b6c27640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f66b015f650 con 0x7f66b0156b90 2024-09-17T03:16:03.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.621+0000 7f66b6c27640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f66b015f4d0 con 0x7f66b00cc590 2024-09-17T03:16:03.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.621+0000 7f66b6c27640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f66b015f7d0 con 0x7f66b015f0c0 2024-09-17T03:16:03.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.622+0000 7f66b51c4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66b0156b90 0x7f66b00c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:03.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.622+0000 7f66b51c4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66b0156b90 0x7f66b00c9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44956/0 (socket says 172.21.15.87:44956) 2024-09-17T03:16:03.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.622+0000 7f66b51c4640 1 -- 172.21.15.87:0/3210599660 learned_addr learned my addr 172.21.15.87:0/3210599660 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:03.622 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.622+0000 7f66b49c3640 1 --2- 172.21.15.87:0/3210599660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66b00cc590 0x7f66b00b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:03.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.622+0000 7f66affff640 1 --2- 172.21.15.87:0/3210599660 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f66b015f0c0 0x7f66b01f33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:03.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.622+0000 7f66b51c4640 1 -- 172.21.15.87:0/3210599660 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f66b015f0c0 msgr2=0x7f66b01f33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:03.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.622+0000 7f66b51c4640 1 --2- 172.21.15.87:0/3210599660 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f66b015f0c0 0x7f66b01f33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:03.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.622+0000 7f66b51c4640 1 -- 172.21.15.87:0/3210599660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66b00cc590 msgr2=0x7f66b00b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:03.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.622+0000 7f66b51c4640 1 --2- 172.21.15.87:0/3210599660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66b00cc590 0x7f66b00b3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:03.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.622+0000 7f66b51c4640 1 -- 172.21.15.87:0/3210599660 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b00c7170 con 0x7f66b0156b90 2024-09-17T03:16:03.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.622+0000 7f66b51c4640 1 --2- 172.21.15.87:0/3210599660 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66b0156b90 0x7f66b00c9470 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009cd0 tx=0x7f66a8032d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:03.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.622+0000 7f66ad7fa640 1 -- 172.21.15.87:0/3210599660 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a8041070 con 0x7f66b0156b90 2024-09-17T03:16:03.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.622+0000 7f66ad7fa640 1 -- 172.21.15.87:0/3210599660 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66a8004450 con 0x7f66b0156b90 2024-09-17T03:16:03.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.622+0000 7f66b6c27640 1 -- 172.21.15.87:0/3210599660 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b00c7450 con 0x7f66b0156b90 2024-09-17T03:16:03.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.622+0000 7f66ad7fa640 1 -- 172.21.15.87:0/3210599660 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a8031430 con 0x7f66b0156b90 2024-09-17T03:16:03.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.622+0000 7f66b6c27640 1 -- 172.21.15.87:0/3210599660 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b0156670 con 0x7f66b0156b90 2024-09-17T03:16:03.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.623+0000 7f66b6c27640 1 -- 172.21.15.87:0/3210599660 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66b00c7de0 con 0x7f66b0156b90 2024-09-17T03:16:03.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.624+0000 7f66ad7fa640 1 -- 172.21.15.87:0/3210599660 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f66a8044070 con 0x7f66b0156b90 2024-09-17T03:16:03.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.625+0000 7f66ad7fa640 1 --2- 172.21.15.87:0/3210599660 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6694078b90 0x7f669407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:03.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.625+0000 7f66ad7fa640 1 -- 172.21.15.87:0/3210599660 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5738+0+0 (secure 0 0 0) 0x7f66a802e020 con 0x7f66b0156b90 2024-09-17T03:16:03.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.625+0000 7f66b49c3640 1 --2- 172.21.15.87:0/3210599660 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6694078b90 0x7f669407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:03.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.626+0000 7f66b49c3640 1 --2- 172.21.15.87:0/3210599660 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6694078b90 0x7f669407b0a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009530 tx=0x7f66a4009260 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:03.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.629+0000 7f66ad7fa640 1 -- 172.21.15.87:0/3210599660 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f66a804b030 con 0x7f66b0156b90 2024-09-17T03:16:03.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.839+0000 7f66b6c27640 1 -- 172.21.15.87:0/3210599660 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66b00c57b0 con 0x7f66b0156b90 2024-09-17T03:16:03.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.839+0000 7f66ad7fa640 1 -- 172.21.15.87:0/3210599660 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5229 (secure 0 0 0) 0x7f66a8083a60 con 0x7f66b0156b90 2024-09-17T03:16:03.838 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:16:03.838 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":20,"btime":"2024-09-17T03:15:41:361821+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5104,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/1606650259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1606650259},{"type":"v1","addr":"172.21.15.87:6834","nonce":1606650259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":5116,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6836/3876173555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3876173555},{"type":"v1","addr":"172.21.15.149:6836","nonce":3876173555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1657329632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":1657329632},{"type":"v1","addr":"172.21.15.87:6835","nonce":1657329632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:15:38.707286+0000","modified":"2024-09-17T03:15:40.712230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5487":{"gid":5487,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6837/3562056427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6835","nonce":3562056427},{"type":"v1","addr":"172.21.15.149:6837","nonce":3562056427}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5487,"qdb_cluster":[5487]},"id":3}]} 2024-09-17T03:16:03.838 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 20 2024-09-17T03:16:03.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.841+0000 7f66b6c27640 1 -- 172.21.15.87:0/3210599660 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6694078b90 msgr2=0x7f669407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:03.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.841+0000 7f66b6c27640 1 --2- 172.21.15.87:0/3210599660 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6694078b90 0x7f669407b0a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009530 tx=0x7f66a4009260 comp rx=0 tx=0).stop 2024-09-17T03:16:03.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.841+0000 7f66b6c27640 1 -- 172.21.15.87:0/3210599660 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66b0156b90 msgr2=0x7f66b00c9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:03.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.841+0000 7f66b6c27640 1 --2- 172.21.15.87:0/3210599660 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66b0156b90 0x7f66b00c9470 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009cd0 tx=0x7f66a8032d30 comp rx=0 tx=0).stop 2024-09-17T03:16:03.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.841+0000 7f66b6c27640 1 -- 172.21.15.87:0/3210599660 shutdown_connections 2024-09-17T03:16:03.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.841+0000 7f66b6c27640 1 --2- 172.21.15.87:0/3210599660 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6694078b90 0x7f669407b0a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:03.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.841+0000 7f66b6c27640 1 --2- 172.21.15.87:0/3210599660 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66b0156b90 0x7f66b00c9470 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:03.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.841+0000 7f66b6c27640 1 --2- 172.21.15.87:0/3210599660 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f66b015f0c0 0x7f66b01f33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:03.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.841+0000 7f66b6c27640 1 --2- 172.21.15.87:0/3210599660 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f66b00cc590 0x7f66b00b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:03.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.842+0000 7f66b6c27640 1 -- 172.21.15.87:0/3210599660 >> 172.21.15.87:0/3210599660 conn(0x7f66b006ee70 msgr2=0x7f66b006f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:03.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.842+0000 7f66b6c27640 1 -- 172.21.15.87:0/3210599660 shutdown_connections 2024-09-17T03:16:03.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.842+0000 7f66b6c27640 1 -- 172.21.15.87:0/3210599660 wait complete. 2024-09-17T03:16:03.852 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:16:03.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.965+0000 7fc7757ba640 1 -- 172.21.15.87:0/2640388911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7700b65b0 msgr2=0x7fc7700d3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:03.964 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.965+0000 7fc7757ba640 1 --2- 172.21.15.87:0/2640388911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7700b65b0 0x7fc7700d3010 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fc768009830 tx=0x7fc76802a5c0 comp rx=0 tx=0).stop 2024-09-17T03:16:03.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.965+0000 7fc7757ba640 1 -- 172.21.15.87:0/2640388911 shutdown_connections 2024-09-17T03:16:03.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.965+0000 7fc7757ba640 1 --2- 172.21.15.87:0/2640388911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7700b65b0 0x7fc7700d3010 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:03.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.965+0000 7fc7757ba640 1 -- 172.21.15.87:0/2640388911 >> 172.21.15.87:0/2640388911 conn(0x7fc770069900 msgr2=0x7fc7700d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:03.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.965+0000 7fc7757ba640 1 -- 172.21.15.87:0/2640388911 shutdown_connections 2024-09-17T03:16:03.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.965+0000 7fc7757ba640 1 -- 172.21.15.87:0/2640388911 wait complete. 2024-09-17T03:16:03.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.967+0000 7fc7757ba640 1 Processor -- start 2024-09-17T03:16:03.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.967+0000 7fc7757ba640 1 -- start start 2024-09-17T03:16:03.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.968+0000 7fc7757ba640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7700cf020 0x7fc7700ca750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:03.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.968+0000 7fc7757ba640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc77015c5f0 0x7fc770151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:03.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.968+0000 7fc7757ba640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc7701f8c70 0x7fc7701f9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:03.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.968+0000 7fc7757ba640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc7700d3550 con 0x7fc7700cf020 2024-09-17T03:16:03.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.968+0000 7fc7757ba640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc7700cf7b0 con 0x7fc77015c5f0 2024-09-17T03:16:03.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.968+0000 7fc7757ba640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc7700d36d0 con 0x7fc7701f8c70 2024-09-17T03:16:03.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.968+0000 7fc76effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7700cf020 0x7fc7700ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:03.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.968+0000 7fc76effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7700cf020 0x7fc7700ca750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44964/0 (socket says 172.21.15.87:44964) 2024-09-17T03:16:03.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.968+0000 7fc76effd640 1 -- 172.21.15.87:0/3183146464 learned_addr learned my addr 172.21.15.87:0/3183146464 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:03.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.968+0000 7fc76f7fe640 1 --2- 172.21.15.87:0/3183146464 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc7701f8c70 0x7fc7701f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:03.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.968+0000 7fc76e7fc640 1 --2- 172.21.15.87:0/3183146464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc77015c5f0 0x7fc770151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:03.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.969+0000 7fc76effd640 1 -- 172.21.15.87:0/3183146464 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc7701f8c70 msgr2=0x7fc7701f9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:03.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.969+0000 7fc76effd640 1 --2- 172.21.15.87:0/3183146464 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc7701f8c70 0x7fc7701f9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:03.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.969+0000 7fc76effd640 1 -- 172.21.15.87:0/3183146464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc77015c5f0 msgr2=0x7fc770151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:03.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.969+0000 7fc76effd640 1 --2- 172.21.15.87:0/3183146464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc77015c5f0 0x7fc770151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:03.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.969+0000 7fc76effd640 1 -- 172.21.15.87:0/3183146464 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7700cd590 con 0x7fc7700cf020 2024-09-17T03:16:03.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.969+0000 7fc76e7fc640 1 --2- 172.21.15.87:0/3183146464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc77015c5f0 0x7fc770151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:16:03.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.969+0000 7fc76effd640 1 --2- 172.21.15.87:0/3183146464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7700cf020 0x7fc7700ca750 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc76000c760 tx=0x7fc76000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:03.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.969+0000 7fc7577fe640 1 -- 172.21.15.87:0/3183146464 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc760003b70 con 0x7fc7700cf020 2024-09-17T03:16:03.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.969+0000 7fc7577fe640 1 -- 172.21.15.87:0/3183146464 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc760003d30 con 0x7fc7700cf020 2024-09-17T03:16:03.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.969+0000 7fc7577fe640 1 -- 172.21.15.87:0/3183146464 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76000aab0 con 0x7fc7700cf020 2024-09-17T03:16:03.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.969+0000 7fc7757ba640 1 -- 172.21.15.87:0/3183146464 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc770151fe0 con 0x7fc7700cf020 2024-09-17T03:16:03.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.969+0000 7fc7757ba640 1 -- 172.21.15.87:0/3183146464 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7700cdb10 con 0x7fc7700cf020 2024-09-17T03:16:03.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.970+0000 7fc7757ba640 1 -- 172.21.15.87:0/3183146464 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7700d4310 con 0x7fc7700cf020 2024-09-17T03:16:03.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.971+0000 7fc7577fe640 1 -- 172.21.15.87:0/3183146464 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc76000ac50 con 0x7fc7700cf020 2024-09-17T03:16:03.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.971+0000 7fc7577fe640 1 --2- 172.21.15.87:0/3183146464 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc75c078bc0 0x7fc75c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:03.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.971+0000 7fc7577fe640 1 -- 172.21.15.87:0/3183146464 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5738+0+0 (secure 0 0 0) 0x7fc76009ac30 con 0x7fc7700cf020 2024-09-17T03:16:03.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.971+0000 7fc76e7fc640 1 --2- 172.21.15.87:0/3183146464 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc75c078bc0 0x7fc75c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:03.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.972+0000 7fc76e7fc640 1 --2- 172.21.15.87:0/3183146464 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc75c078bc0 0x7fc75c07b0d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc7700d14d0 tx=0x7fc764009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:03.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:03.976+0000 7fc7577fe640 1 -- 172.21.15.87:0/3183146464 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc760063050 con 0x7fc7700cf020 2024-09-17T03:16:04.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.151+0000 7fc7757ba640 1 -- 172.21.15.87:0/3183146464 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7700b6ad0 con 0x7fc7700cf020 2024-09-17T03:16:04.150 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.152+0000 7fc7577fe640 1 -- 172.21.15.87:0/3183146464 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16742 (secure 0 0 0) 0x7fc760062950 con 0x7fc7700cf020 2024-09-17T03:16:04.151 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:16:04.151 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":52,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:15:54.519051+0000","last_up_change":"2024-09-17T03:14:01.833605+0000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:14:03.433404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:15:36.355536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-17T03:15:37.674416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6825","nonce":2430409749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6827","nonce":2430409749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6831","nonce":2430409749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6829","nonce":2430409749}]},"public_addr":"172.21.15.87:6825/2430409749","cluster_addr":"172.21.15.87:6827/2430409749","heartbeat_back_addr":"172.21.15.87:6831/2430409749","heartbeat_front_addr":"172.21.15.87:6829/2430409749","state":["exists","up"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6809","nonce":3467605435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6811","nonce":3467605435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6815","nonce":3467605435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6813","nonce":3467605435}]},"public_addr":"172.21.15.87:6809/3467605435","cluster_addr":"172.21.15.87:6811/3467605435","heartbeat_back_addr":"172.21.15.87:6815/3467605435","heartbeat_front_addr":"172.21.15.87:6813/3467605435","state":["exists","up"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6801","nonce":1450834766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6803","nonce":1450834766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6807","nonce":1450834766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6805","nonce":1450834766}]},"public_addr":"172.21.15.87:6801/1450834766","cluster_addr":"172.21.15.87:6803/1450834766","heartbeat_back_addr":"172.21.15.87:6807/1450834766","heartbeat_front_addr":"172.21.15.87:6805/1450834766","state":["exists","up"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6817","nonce":4253161216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6819","nonce":4253161216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6823","nonce":4253161216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6821","nonce":4253161216}]},"public_addr":"172.21.15.87:6817/4253161216","cluster_addr":"172.21.15.87:6819/4253161216","heartbeat_back_addr":"172.21.15.87:6823/4253161216","heartbeat_front_addr":"172.21.15.87:6821/4253161216","state":["exists","up"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6803","nonce":2414907089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6805","nonce":2414907089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6809","nonce":2414907089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6807","nonce":2414907089}]},"public_addr":"172.21.15.149:6803/2414907089","cluster_addr":"172.21.15.149:6805/2414907089","heartbeat_back_addr":"172.21.15.149:6809/2414907089","heartbeat_front_addr":"172.21.15.149:6807/2414907089","state":["exists","up"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6827","nonce":2182999530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6829","nonce":2182999530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6833","nonce":2182999530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6831","nonce":2182999530}]},"public_addr":"172.21.15.149:6827/2182999530","cluster_addr":"172.21.15.149:6829/2182999530","heartbeat_back_addr":"172.21.15.149:6833/2182999530","heartbeat_front_addr":"172.21.15.149:6831/2182999530","state":["exists","up"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6811","nonce":4289519146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6813","nonce":4289519146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6817","nonce":4289519146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6815","nonce":4289519146}]},"public_addr":"172.21.15.149:6811/4289519146","cluster_addr":"172.21.15.149:6813/4289519146","heartbeat_back_addr":"172.21.15.149:6817/4289519146","heartbeat_front_addr":"172.21.15.149:6815/4289519146","state":["exists","up"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6819","nonce":2771010477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6821","nonce":2771010477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6825","nonce":2771010477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6823","nonce":2771010477}]},"public_addr":"172.21.15.149:6819/2771010477","cluster_addr":"172.21.15.149:6821/2771010477","heartbeat_back_addr":"172.21.15.149:6825/2771010477","heartbeat_front_addr":"172.21.15.149:6823/2771010477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.313297+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.642861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.654781+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.302606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.385791+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.915266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.121365+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.394432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.31","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:16:04.153 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.154+0000 7fc7757ba640 1 -- 172.21.15.87:0/3183146464 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc75c078bc0 msgr2=0x7fc75c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:04.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.154+0000 7fc7757ba640 1 --2- 172.21.15.87:0/3183146464 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc75c078bc0 0x7fc75c07b0d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc7700d14d0 tx=0x7fc764009040 comp rx=0 tx=0).stop 2024-09-17T03:16:04.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.154+0000 7fc7757ba640 1 -- 172.21.15.87:0/3183146464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7700cf020 msgr2=0x7fc7700ca750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:04.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.154+0000 7fc7757ba640 1 --2- 172.21.15.87:0/3183146464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7700cf020 0x7fc7700ca750 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc76000c760 tx=0x7fc76000cc20 comp rx=0 tx=0).stop 2024-09-17T03:16:04.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.154+0000 7fc7757ba640 1 -- 172.21.15.87:0/3183146464 shutdown_connections 2024-09-17T03:16:04.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.154+0000 7fc7757ba640 1 --2- 172.21.15.87:0/3183146464 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc75c078bc0 0x7fc75c07b0d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:04.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.154+0000 7fc7757ba640 1 --2- 172.21.15.87:0/3183146464 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc7701f8c70 0x7fc7701f9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:04.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.154+0000 7fc7757ba640 1 --2- 172.21.15.87:0/3183146464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc77015c5f0 0x7fc770151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:04.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.154+0000 7fc7757ba640 1 --2- 172.21.15.87:0/3183146464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7700cf020 0x7fc7700ca750 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:04.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.154+0000 7fc7757ba640 1 -- 172.21.15.87:0/3183146464 >> 172.21.15.87:0/3183146464 conn(0x7fc770069900 msgr2=0x7fc7700d1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:04.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.154+0000 7fc7757ba640 1 -- 172.21.15.87:0/3183146464 shutdown_connections 2024-09-17T03:16:04.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.154+0000 7fc7757ba640 1 -- 172.21.15.87:0/3183146464 wait complete. 2024-09-17T03:16:04.164 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T03:16:04.165 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:16:04.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.278+0000 7f27a5230640 1 -- 172.21.15.87:0/2288800301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27a00cf430 msgr2=0x7f27a00b63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:04.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.278+0000 7f27a5230640 1 --2- 172.21.15.87:0/2288800301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27a00cf430 0x7f27a00b63e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f27a00cfef0 tx=0x7f279802aa00 comp rx=0 tx=0).stop 2024-09-17T03:16:04.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.278+0000 7f27a5230640 1 -- 172.21.15.87:0/2288800301 shutdown_connections 2024-09-17T03:16:04.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.278+0000 7f27a5230640 1 --2- 172.21.15.87:0/2288800301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27a00cf430 0x7f27a00b63e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:04.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.278+0000 7f27a5230640 1 --2- 172.21.15.87:0/2288800301 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f27a00cf020 0x7f27a00b5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:04.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.278+0000 7f27a5230640 1 --2- 172.21.15.87:0/2288800301 >> v2:172.21.15.149:3300/0 conn(0x7f27a015c5f0 0x7f27a015f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:04.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.278+0000 7f27a5230640 1 -- 172.21.15.87:0/2288800301 >> 172.21.15.87:0/2288800301 conn(0x7f27a0069900 msgr2=0x7f27a00d4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:04.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.278+0000 7f27a5230640 1 -- 172.21.15.87:0/2288800301 shutdown_connections 2024-09-17T03:16:04.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.278+0000 7f27a5230640 1 -- 172.21.15.87:0/2288800301 wait complete. 2024-09-17T03:16:04.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.280+0000 7f27a5230640 1 Processor -- start 2024-09-17T03:16:04.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.280+0000 7f27a5230640 1 -- start start 2024-09-17T03:16:04.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.281+0000 7f27a5230640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f27a00cf020 0x7f27a00c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:04.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.281+0000 7f27a5230640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27a00cf430 0x7f27a0148fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:04.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.281+0000 7f27a5230640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27a015c5f0 0x7f27a015dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:04.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.281+0000 7f27a5230640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f27a00d3190 con 0x7f27a00cf430 2024-09-17T03:16:04.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.281+0000 7f27a5230640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f27a00d3010 con 0x7f27a015c5f0 2024-09-17T03:16:04.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.281+0000 7f27a5230640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f27a00d3310 con 0x7f27a00cf020 2024-09-17T03:16:04.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.281+0000 7f279e59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27a00cf430 0x7f27a0148fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:04.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.281+0000 7f279e59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27a00cf430 0x7f27a0148fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44992/0 (socket says 172.21.15.87:44992) 2024-09-17T03:16:04.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.281+0000 7f279e59c640 1 -- 172.21.15.87:0/1413164713 learned_addr learned my addr 172.21.15.87:0/1413164713 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:04.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.282+0000 7f279f59e640 1 --2- 172.21.15.87:0/1413164713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27a015c5f0 0x7f27a015dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:04.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.282+0000 7f279ed9d640 1 --2- 172.21.15.87:0/1413164713 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f27a00cf020 0x7f27a00c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:04.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.282+0000 7f279e59c640 1 -- 172.21.15.87:0/1413164713 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f27a00cf020 msgr2=0x7f27a00c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:04.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.282+0000 7f279e59c640 1 --2- 172.21.15.87:0/1413164713 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f27a00cf020 0x7f27a00c80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:04.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.282+0000 7f279e59c640 1 -- 172.21.15.87:0/1413164713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27a015c5f0 msgr2=0x7f27a015dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:04.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.282+0000 7f279e59c640 1 --2- 172.21.15.87:0/1413164713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27a015c5f0 0x7f27a015dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:04.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.282+0000 7f279e59c640 1 -- 172.21.15.87:0/1413164713 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a00c8630 con 0x7f27a00cf430 2024-09-17T03:16:04.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.282+0000 7f279e59c640 1 --2- 172.21.15.87:0/1413164713 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27a00cf430 0x7f27a0148fe0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f279400c940 tx=0x7f279400ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:04.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.282+0000 7f27877fe640 1 -- 172.21.15.87:0/1413164713 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2794007bb0 con 0x7f27a00cf430 2024-09-17T03:16:04.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.282+0000 7f27877fe640 1 -- 172.21.15.87:0/1413164713 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2794007e40 con 0x7f27a00cf430 2024-09-17T03:16:04.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.282+0000 7f27a5230640 1 -- 172.21.15.87:0/1413164713 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a01f5790 con 0x7f27a00cf430 2024-09-17T03:16:04.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.282+0000 7f27877fe640 1 -- 172.21.15.87:0/1413164713 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279400f670 con 0x7f27a00cf430 2024-09-17T03:16:04.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.282+0000 7f27a5230640 1 -- 172.21.15.87:0/1413164713 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a01f5d50 con 0x7f27a00cf430 2024-09-17T03:16:04.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.283+0000 7f27a5230640 1 -- 172.21.15.87:0/1413164713 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27a015f890 con 0x7f27a00cf430 2024-09-17T03:16:04.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.284+0000 7f27877fe640 1 -- 172.21.15.87:0/1413164713 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2794018400 con 0x7f27a00cf430 2024-09-17T03:16:04.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.284+0000 7f27877fe640 1 --2- 172.21.15.87:0/1413164713 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f278c078bc0 0x7f278c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:04.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.284+0000 7f27877fe640 1 -- 172.21.15.87:0/1413164713 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5738+0+0 (secure 0 0 0) 0x7f279409b490 con 0x7f27a00cf430 2024-09-17T03:16:04.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.285+0000 7f279ed9d640 1 --2- 172.21.15.87:0/1413164713 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f278c078bc0 0x7f278c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:04.286 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.285+0000 7f279ed9d640 1 --2- 172.21.15.87:0/1413164713 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f278c078bc0 0x7f278c07b0d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f279000b9a0 tx=0x7f2790005c20 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:04.287 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.289+0000 7f27877fe640 1 -- 172.21.15.87:0/1413164713 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f2794014030 con 0x7f27a00cf430 2024-09-17T03:16:04.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.497+0000 7f27a5230640 1 -- 172.21.15.87:0/1413164713 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27a01f5520 con 0x7f27a00cf430 2024-09-17T03:16:04.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.498+0000 7f27877fe640 1 -- 172.21.15.87:0/1413164713 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5229 (secure 0 0 0) 0x7f27940638b0 con 0x7f27a00cf430 2024-09-17T03:16:04.497 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:16:04.497 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":20,"btime":"2024-09-17T03:15:41:361821+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5104,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/1606650259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1606650259},{"type":"v1","addr":"172.21.15.87:6834","nonce":1606650259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":5116,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6836/3876173555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3876173555},{"type":"v1","addr":"172.21.15.149:6836","nonce":3876173555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1657329632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":1657329632},{"type":"v1","addr":"172.21.15.87:6835","nonce":1657329632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:15:38.707286+0000","modified":"2024-09-17T03:15:40.712230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5487":{"gid":5487,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6837/3562056427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6835","nonce":3562056427},{"type":"v1","addr":"172.21.15.149:6837","nonce":3562056427}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5487,"qdb_cluster":[5487]},"id":3}]} 2024-09-17T03:16:04.497 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 20 2024-09-17T03:16:04.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.500+0000 7f27a5230640 1 -- 172.21.15.87:0/1413164713 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f278c078bc0 msgr2=0x7f278c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:04.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.500+0000 7f27a5230640 1 --2- 172.21.15.87:0/1413164713 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f278c078bc0 0x7f278c07b0d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f279000b9a0 tx=0x7f2790005c20 comp rx=0 tx=0).stop 2024-09-17T03:16:04.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.500+0000 7f27a5230640 1 -- 172.21.15.87:0/1413164713 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27a00cf430 msgr2=0x7f27a0148fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:04.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.500+0000 7f27a5230640 1 --2- 172.21.15.87:0/1413164713 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27a00cf430 0x7f27a0148fe0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f279400c940 tx=0x7f279400ce00 comp rx=0 tx=0).stop 2024-09-17T03:16:04.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.500+0000 7f27a5230640 1 -- 172.21.15.87:0/1413164713 shutdown_connections 2024-09-17T03:16:04.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.500+0000 7f27a5230640 1 --2- 172.21.15.87:0/1413164713 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f278c078bc0 0x7f278c07b0d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:04.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.500+0000 7f27a5230640 1 --2- 172.21.15.87:0/1413164713 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f27a015c5f0 0x7f27a015dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:04.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.500+0000 7f27a5230640 1 --2- 172.21.15.87:0/1413164713 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f27a00cf430 0x7f27a0148fe0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:04.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.500+0000 7f27a5230640 1 --2- 172.21.15.87:0/1413164713 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f27a00cf020 0x7f27a00c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:04.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.500+0000 7f27a5230640 1 -- 172.21.15.87:0/1413164713 >> 172.21.15.87:0/1413164713 conn(0x7f27a0069900 msgr2=0x7f27a0161c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:04.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.500+0000 7f27a5230640 1 -- 172.21.15.87:0/1413164713 shutdown_connections 2024-09-17T03:16:04.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.500+0000 7f27a5230640 1 -- 172.21.15.87:0/1413164713 wait complete. 2024-09-17T03:16:04.511 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:16:04.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.624+0000 7f541d04a640 1 -- 172.21.15.87:0/2828475494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5400003a30 msgr2=0x7f5400003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:04.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.624+0000 7f541d04a640 1 --2- 172.21.15.87:0/2828475494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5400003a30 0x7f5400003ea0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f540800c610 tx=0x7f5400005830 comp rx=0 tx=0).stop 2024-09-17T03:16:04.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.624+0000 7f541759e640 1 -- 172.21.15.87:0/2828475494 reap_dead start 2024-09-17T03:16:04.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.624+0000 7f541d04a640 1 -- 172.21.15.87:0/2828475494 shutdown_connections 2024-09-17T03:16:04.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.624+0000 7f541d04a640 1 -- 172.21.15.87:0/2828475494 >> 172.21.15.87:0/2828475494 conn(0x7f5418069330 msgr2=0x7f54180ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:04.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.624+0000 7f541d04a640 1 -- 172.21.15.87:0/2828475494 shutdown_connections 2024-09-17T03:16:04.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.624+0000 7f541d04a640 1 -- 172.21.15.87:0/2828475494 wait complete. 2024-09-17T03:16:04.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.626+0000 7f541d04a640 1 Processor -- start 2024-09-17T03:16:04.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.626+0000 7f541d04a640 1 -- start start 2024-09-17T03:16:04.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.628+0000 7f541d04a640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5400003a30 0x7f5418148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:04.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.628+0000 7f541d04a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54180a94a0 0x7f541815a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:04.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.628+0000 7f541d04a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54180a98b0 0x7f541815a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:04.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.628+0000 7f541d04a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f54180d1c40 con 0x7f54180a98b0 2024-09-17T03:16:04.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.628+0000 7f541d04a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f54180b7d20 con 0x7f54180a94a0 2024-09-17T03:16:04.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.628+0000 7f541d04a640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f54180d0940 con 0x7f5400003a30 2024-09-17T03:16:04.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.628+0000 7f541759e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54180a98b0 0x7f541815a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:04.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.628+0000 7f541759e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54180a98b0 0x7f541815a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45012/0 (socket says 172.21.15.87:45012) 2024-09-17T03:16:04.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.628+0000 7f541759e640 1 -- 172.21.15.87:0/4074553668 learned_addr learned my addr 172.21.15.87:0/4074553668 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:04.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.628+0000 7f5416d9d640 1 --2- 172.21.15.87:0/4074553668 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5400003a30 0x7f5418148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:04.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.628+0000 7f541659c640 1 --2- 172.21.15.87:0/4074553668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54180a94a0 0x7f541815a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:04.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.628+0000 7f541759e640 1 -- 172.21.15.87:0/4074553668 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5400003a30 msgr2=0x7f5418148410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:04.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.628+0000 7f541759e640 1 --2- 172.21.15.87:0/4074553668 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5400003a30 0x7f5418148410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:04.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.628+0000 7f541759e640 1 -- 172.21.15.87:0/4074553668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54180a94a0 msgr2=0x7f541815a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:04.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.628+0000 7f541759e640 1 --2- 172.21.15.87:0/4074553668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54180a94a0 0x7f541815a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:04.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.628+0000 7f541759e640 1 -- 172.21.15.87:0/4074553668 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5418153310 con 0x7f54180a98b0 2024-09-17T03:16:04.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.628+0000 7f541759e640 1 --2- 172.21.15.87:0/4074553668 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54180a98b0 0x7f541815a650 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f54180cdd30 tx=0x7f5410011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:04.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.628+0000 7f53ff7fe640 1 -- 172.21.15.87:0/4074553668 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5410004230 con 0x7f54180a98b0 2024-09-17T03:16:04.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.628+0000 7f53ff7fe640 1 -- 172.21.15.87:0/4074553668 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5410016070 con 0x7f54180a98b0 2024-09-17T03:16:04.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.628+0000 7f541d04a640 1 -- 172.21.15.87:0/4074553668 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54181535f0 con 0x7f54180a98b0 2024-09-17T03:16:04.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.628+0000 7f53ff7fe640 1 -- 172.21.15.87:0/4074553668 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541000fac0 con 0x7f54180a98b0 2024-09-17T03:16:04.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.629+0000 7f541d04a640 1 -- 172.21.15.87:0/4074553668 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5418153bb0 con 0x7f54180a98b0 2024-09-17T03:16:04.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.630+0000 7f541d04a640 1 -- 172.21.15.87:0/4074553668 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54180d0e50 con 0x7f54180a98b0 2024-09-17T03:16:04.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.630+0000 7f53ff7fe640 1 -- 172.21.15.87:0/4074553668 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f541000fc60 con 0x7f54180a98b0 2024-09-17T03:16:04.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.631+0000 7f53ff7fe640 1 --2- 172.21.15.87:0/4074553668 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5404078b90 0x7f540407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:04.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.631+0000 7f53ff7fe640 1 -- 172.21.15.87:0/4074553668 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5738+0+0 (secure 0 0 0) 0x7f541009b580 con 0x7f54180a98b0 2024-09-17T03:16:04.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.631+0000 7f5416d9d640 1 --2- 172.21.15.87:0/4074553668 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5404078b90 0x7f540407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:04.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.632+0000 7f5416d9d640 1 --2- 172.21.15.87:0/4074553668 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5404078b90 0x7f540407b0a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5408033000 tx=0x7f5408001360 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:04.634 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.635+0000 7f53ff7fe640 1 -- 172.21.15.87:0/4074553668 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f5410063350 con 0x7f54180a98b0 2024-09-17T03:16:04.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.845+0000 7f541d04a640 1 -- 172.21.15.87:0/4074553668 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5418148950 con 0x7f54180a98b0 2024-09-17T03:16:04.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.846+0000 7f53ff7fe640 1 -- 172.21.15.87:0/4074553668 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+5229 (secure 0 0 0) 0x7f5410067d80 con 0x7f54180a98b0 2024-09-17T03:16:04.845 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:16:04.845 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":20,"btime":"2024-09-17T03:15:41:361821+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5104,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/1606650259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1606650259},{"type":"v1","addr":"172.21.15.87:6834","nonce":1606650259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":5116,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6836/3876173555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3876173555},{"type":"v1","addr":"172.21.15.149:6836","nonce":3876173555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1657329632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":1657329632},{"type":"v1","addr":"172.21.15.87:6835","nonce":1657329632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-17T03:15:38.707286+0000","modified":"2024-09-17T03:15:40.712230+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5487},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5487":{"gid":5487,"name":"b","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.149:6837/3562056427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6835","nonce":3562056427},{"type":"v1","addr":"172.21.15.149:6837","nonce":3562056427}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5487,"qdb_cluster":[5487]},"id":3}]} 2024-09-17T03:16:04.845 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 20 2024-09-17T03:16:04.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.848+0000 7f541d04a640 1 -- 172.21.15.87:0/4074553668 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5404078b90 msgr2=0x7f540407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:04.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.848+0000 7f541d04a640 1 --2- 172.21.15.87:0/4074553668 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5404078b90 0x7f540407b0a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5408033000 tx=0x7f5408001360 comp rx=0 tx=0).stop 2024-09-17T03:16:04.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.848+0000 7f541d04a640 1 -- 172.21.15.87:0/4074553668 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54180a98b0 msgr2=0x7f541815a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:04.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.848+0000 7f541d04a640 1 --2- 172.21.15.87:0/4074553668 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54180a98b0 0x7f541815a650 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f54180cdd30 tx=0x7f5410011b10 comp rx=0 tx=0).stop 2024-09-17T03:16:04.847 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.848+0000 7f541d04a640 1 -- 172.21.15.87:0/4074553668 shutdown_connections 2024-09-17T03:16:04.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.848+0000 7f541d04a640 1 --2- 172.21.15.87:0/4074553668 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5404078b90 0x7f540407b0a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:04.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.848+0000 7f541d04a640 1 --2- 172.21.15.87:0/4074553668 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54180a98b0 0x7f541815a650 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:04.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.848+0000 7f541d04a640 1 --2- 172.21.15.87:0/4074553668 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f54180a94a0 0x7f541815a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:04.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.848+0000 7f541d04a640 1 --2- 172.21.15.87:0/4074553668 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f5400003a30 0x7f5418148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:04.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.848+0000 7f541d04a640 1 -- 172.21.15.87:0/4074553668 >> 172.21.15.87:0/4074553668 conn(0x7f5418069330 msgr2=0x7f541815cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:04.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.848+0000 7f541d04a640 1 -- 172.21.15.87:0/4074553668 shutdown_connections 2024-09-17T03:16:04.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.848+0000 7f541d04a640 1 -- 172.21.15.87:0/4074553668 wait complete. 2024-09-17T03:16:04.858 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:16:04.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.971+0000 7fdfb5b20640 1 -- 172.21.15.87:0/2347264249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf9c003a00 msgr2=0x7fdf9c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:04.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.971+0000 7fdfb5b20640 1 --2- 172.21.15.87:0/2347264249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf9c003a00 0x7fdf9c003e70 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0009d20 tx=0x7fdf9c005830 comp rx=0 tx=0).stop 2024-09-17T03:16:04.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.971+0000 7fdfaffff640 1 -- 172.21.15.87:0/2347264249 reap_dead start 2024-09-17T03:16:04.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.971+0000 7fdfb5b20640 1 -- 172.21.15.87:0/2347264249 shutdown_connections 2024-09-17T03:16:04.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.971+0000 7fdfb5b20640 1 -- 172.21.15.87:0/2347264249 >> 172.21.15.87:0/2347264249 conn(0x7fdfb0069360 msgr2=0x7fdfb00ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:04.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.971+0000 7fdfb5b20640 1 -- 172.21.15.87:0/2347264249 shutdown_connections 2024-09-17T03:16:04.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.971+0000 7fdfb5b20640 1 -- 172.21.15.87:0/2347264249 wait complete. 2024-09-17T03:16:04.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.973+0000 7fdfb5b20640 1 Processor -- start 2024-09-17T03:16:04.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.973+0000 7fdfb5b20640 1 -- start start 2024-09-17T03:16:04.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.974+0000 7fdfb5b20640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf9c003a00 0x7fdfb0158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:04.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.974+0000 7fdfb5b20640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdfb0159530 0x7fdfb00b2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:04.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.974+0000 7fdfb5b20640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfb00c8100 0x7fdfb00b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:04.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.974+0000 7fdfb5b20640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb00d1470 con 0x7fdf9c003a00 2024-09-17T03:16:04.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.974+0000 7fdfb5b20640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb00b80d0 con 0x7fdfb00c8100 2024-09-17T03:16:04.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.974+0000 7fdfb5b20640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fdfb00d15f0 con 0x7fdfb0159530 2024-09-17T03:16:04.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.974+0000 7fdfaf7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf9c003a00 0x7fdfb0158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:04.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.974+0000 7fdfaf7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf9c003a00 0x7fdfb0158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45028/0 (socket says 172.21.15.87:45028) 2024-09-17T03:16:04.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.974+0000 7fdfaf7fe640 1 -- 172.21.15.87:0/1880576636 learned_addr learned my addr 172.21.15.87:0/1880576636 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:04.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.974+0000 7fdfa7fff640 1 --2- 172.21.15.87:0/1880576636 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdfb0159530 0x7fdfb00b2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:04.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.974+0000 7fdfaffff640 1 --2- 172.21.15.87:0/1880576636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfb00c8100 0x7fdfb00b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:04.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.974+0000 7fdfaf7fe640 1 -- 172.21.15.87:0/1880576636 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdfb0159530 msgr2=0x7fdfb00b2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:04.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.974+0000 7fdfaf7fe640 1 --2- 172.21.15.87:0/1880576636 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdfb0159530 0x7fdfb00b2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:04.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.974+0000 7fdfaf7fe640 1 -- 172.21.15.87:0/1880576636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfb00c8100 msgr2=0x7fdfb00b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:04.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.974+0000 7fdfaf7fe640 1 --2- 172.21.15.87:0/1880576636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfb00c8100 0x7fdfb00b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:04.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.974+0000 7fdfaf7fe640 1 -- 172.21.15.87:0/1880576636 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb0156fa0 con 0x7fdf9c003a00 2024-09-17T03:16:04.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.975+0000 7fdfaf7fe640 1 --2- 172.21.15.87:0/1880576636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf9c003a00 0x7fdfb0158ff0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0009c40 tx=0x7fdfa0003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:04.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.975+0000 7fdfacff9640 1 -- 172.21.15.87:0/1880576636 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa0041020 con 0x7fdf9c003a00 2024-09-17T03:16:04.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.975+0000 7fdfacff9640 1 -- 172.21.15.87:0/1880576636 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfa002ba80 con 0x7fdf9c003a00 2024-09-17T03:16:04.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.975+0000 7fdfb5b20640 1 -- 172.21.15.87:0/1880576636 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb01f5ee0 con 0x7fdf9c003a00 2024-09-17T03:16:04.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.975+0000 7fdfacff9640 1 -- 172.21.15.87:0/1880576636 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa00098f0 con 0x7fdf9c003a00 2024-09-17T03:16:04.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.975+0000 7fdfb5b20640 1 -- 172.21.15.87:0/1880576636 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb01f64a0 con 0x7fdf9c003a00 2024-09-17T03:16:04.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.976+0000 7fdfb5b20640 1 -- 172.21.15.87:0/1880576636 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfb0071ce0 con 0x7fdf9c003a00 2024-09-17T03:16:04.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.977+0000 7fdfacff9640 1 -- 172.21.15.87:0/1880576636 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdfa0044020 con 0x7fdf9c003a00 2024-09-17T03:16:04.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.977+0000 7fdfacff9640 1 --2- 172.21.15.87:0/1880576636 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdf90078bc0 0x7fdf9007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:04.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.977+0000 7fdfacff9640 1 -- 172.21.15.87:0/1880576636 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5738+0+0 (secure 0 0 0) 0x7fdfa00bbb00 con 0x7fdf9c003a00 2024-09-17T03:16:04.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.977+0000 7fdfa7fff640 1 --2- 172.21.15.87:0/1880576636 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdf90078bc0 0x7fdf9007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:04.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.978+0000 7fdfa7fff640 1 --2- 172.21.15.87:0/1880576636 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdf90078bc0 0x7fdf9007b0d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009460 tx=0x7fdf98009390 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:04.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:04.981+0000 7fdfacff9640 1 -- 172.21.15.87:0/1880576636 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fdfa004b070 con 0x7fdf9c003a00 2024-09-17T03:16:05.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.156+0000 7fdfb5b20640 1 -- 172.21.15.87:0/1880576636 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfb00cf1d0 con 0x7fdf9c003a00 2024-09-17T03:16:05.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.157+0000 7fdfacff9640 1 -- 172.21.15.87:0/1880576636 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16742 (secure 0 0 0) 0x7fdfa0003e60 con 0x7fdf9c003a00 2024-09-17T03:16:05.156 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:16:05.156 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":52,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:15:54.519051+0000","last_up_change":"2024-09-17T03:14:01.833605+0000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:14:03.433404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-17T03:15:36.355536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-17T03:15:37.674416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6825","nonce":2430409749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6827","nonce":2430409749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6831","nonce":2430409749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6829","nonce":2430409749}]},"public_addr":"172.21.15.87:6825/2430409749","cluster_addr":"172.21.15.87:6827/2430409749","heartbeat_back_addr":"172.21.15.87:6831/2430409749","heartbeat_front_addr":"172.21.15.87:6829/2430409749","state":["exists","up"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6809","nonce":3467605435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6811","nonce":3467605435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6815","nonce":3467605435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6813","nonce":3467605435}]},"public_addr":"172.21.15.87:6809/3467605435","cluster_addr":"172.21.15.87:6811/3467605435","heartbeat_back_addr":"172.21.15.87:6815/3467605435","heartbeat_front_addr":"172.21.15.87:6813/3467605435","state":["exists","up"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6801","nonce":1450834766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6803","nonce":1450834766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6807","nonce":1450834766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6805","nonce":1450834766}]},"public_addr":"172.21.15.87:6801/1450834766","cluster_addr":"172.21.15.87:6803/1450834766","heartbeat_back_addr":"172.21.15.87:6807/1450834766","heartbeat_front_addr":"172.21.15.87:6805/1450834766","state":["exists","up"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6817","nonce":4253161216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6819","nonce":4253161216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6823","nonce":4253161216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6821","nonce":4253161216}]},"public_addr":"172.21.15.87:6817/4253161216","cluster_addr":"172.21.15.87:6819/4253161216","heartbeat_back_addr":"172.21.15.87:6823/4253161216","heartbeat_front_addr":"172.21.15.87:6821/4253161216","state":["exists","up"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6803","nonce":2414907089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6805","nonce":2414907089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6809","nonce":2414907089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6807","nonce":2414907089}]},"public_addr":"172.21.15.149:6803/2414907089","cluster_addr":"172.21.15.149:6805/2414907089","heartbeat_back_addr":"172.21.15.149:6809/2414907089","heartbeat_front_addr":"172.21.15.149:6807/2414907089","state":["exists","up"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6827","nonce":2182999530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6829","nonce":2182999530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6833","nonce":2182999530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6831","nonce":2182999530}]},"public_addr":"172.21.15.149:6827/2182999530","cluster_addr":"172.21.15.149:6829/2182999530","heartbeat_back_addr":"172.21.15.149:6833/2182999530","heartbeat_front_addr":"172.21.15.149:6831/2182999530","state":["exists","up"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6811","nonce":4289519146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6813","nonce":4289519146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6817","nonce":4289519146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6815","nonce":4289519146}]},"public_addr":"172.21.15.149:6811/4289519146","cluster_addr":"172.21.15.149:6813/4289519146","heartbeat_back_addr":"172.21.15.149:6817/4289519146","heartbeat_front_addr":"172.21.15.149:6815/4289519146","state":["exists","up"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6819","nonce":2771010477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6821","nonce":2771010477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6825","nonce":2771010477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6823","nonce":2771010477}]},"public_addr":"172.21.15.149:6819/2771010477","cluster_addr":"172.21.15.149:6821/2771010477","heartbeat_back_addr":"172.21.15.149:6825/2771010477","heartbeat_front_addr":"172.21.15.149:6823/2771010477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.313297+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.642861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.654781+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.302606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.385791+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.915266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.121365+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.394432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.31","mappings":[{"from":2,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:16:05.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.159+0000 7fdfb5b20640 1 -- 172.21.15.87:0/1880576636 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdf90078bc0 msgr2=0x7fdf9007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:05.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.159+0000 7fdfb5b20640 1 --2- 172.21.15.87:0/1880576636 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdf90078bc0 0x7fdf9007b0d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fdf98009460 tx=0x7fdf98009390 comp rx=0 tx=0).stop 2024-09-17T03:16:05.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.159+0000 7fdfb5b20640 1 -- 172.21.15.87:0/1880576636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf9c003a00 msgr2=0x7fdfb0158ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:05.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.159+0000 7fdfb5b20640 1 --2- 172.21.15.87:0/1880576636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf9c003a00 0x7fdfb0158ff0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0009c40 tx=0x7fdfa0003820 comp rx=0 tx=0).stop 2024-09-17T03:16:05.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.159+0000 7fdfb5b20640 1 -- 172.21.15.87:0/1880576636 shutdown_connections 2024-09-17T03:16:05.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.159+0000 7fdfb5b20640 1 --2- 172.21.15.87:0/1880576636 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fdf90078bc0 0x7fdf9007b0d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:05.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.159+0000 7fdfb5b20640 1 --2- 172.21.15.87:0/1880576636 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fdfb00c8100 0x7fdfb00b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:05.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.159+0000 7fdfb5b20640 1 --2- 172.21.15.87:0/1880576636 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fdfb0159530 0x7fdfb00b2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:05.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.159+0000 7fdfb5b20640 1 --2- 172.21.15.87:0/1880576636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf9c003a00 0x7fdfb0158ff0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:05.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.159+0000 7fdfb5b20640 1 -- 172.21.15.87:0/1880576636 >> 172.21.15.87:0/1880576636 conn(0x7fdfb0069360 msgr2=0x7fdfb00d1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:05.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.159+0000 7fdfb5b20640 1 -- 172.21.15.87:0/1880576636 shutdown_connections 2024-09-17T03:16:05.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.159+0000 7fdfb5b20640 1 -- 172.21.15.87:0/1880576636 wait complete. 2024-09-17T03:16:05.170 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-17T03:16:05.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.284+0000 7f16f424d640 1 -- 172.21.15.87:0/2329581506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16ec0d0c90 msgr2=0x7f16ec160be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:05.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.284+0000 7f16f424d640 1 --2- 172.21.15.87:0/2329581506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16ec0d0c90 0x7f16ec160be0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f16e8009830 tx=0x7f16e802a4a0 comp rx=0 tx=0).stop 2024-09-17T03:16:05.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.284+0000 7f16f424d640 1 -- 172.21.15.87:0/2329581506 shutdown_connections 2024-09-17T03:16:05.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.284+0000 7f16f424d640 1 --2- 172.21.15.87:0/2329581506 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f16ec15cb10 0x7f16ec15cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:05.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.284+0000 7f16f424d640 1 --2- 172.21.15.87:0/2329581506 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16ec0d0c90 0x7f16ec160be0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:05.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.284+0000 7f16f424d640 1 -- 172.21.15.87:0/2329581506 >> 172.21.15.87:0/2329581506 conn(0x7f16ec0d1180 msgr2=0x7f16ec0d1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:05.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.284+0000 7f16f424d640 1 -- 172.21.15.87:0/2329581506 shutdown_connections 2024-09-17T03:16:05.284 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.284+0000 7f16f424d640 1 -- 172.21.15.87:0/2329581506 wait complete. 2024-09-17T03:16:05.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.286+0000 7f16f424d640 1 Processor -- start 2024-09-17T03:16:05.285 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.286+0000 7f16f424d640 1 -- start start 2024-09-17T03:16:05.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.287+0000 7f16f424d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16ec06cde0 0x7f16ec148650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:05.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.287+0000 7f16f424d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16ec0d0c90 0x7f16ec14a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:05.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.287+0000 7f16f424d640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f16ec15cb10 0x7f16ec1f6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:05.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.287+0000 7f16f424d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec0cdf80 con 0x7f16ec06cde0 2024-09-17T03:16:05.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.287+0000 7f16f424d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f16ec0cde00 con 0x7f16ec0d0c90 2024-09-17T03:16:05.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.287+0000 7f16f424d640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f16ec0ce100 con 0x7f16ec15cb10 2024-09-17T03:16:05.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.287+0000 7f16f1fe9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16ec06cde0 0x7f16ec148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:05.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.288+0000 7f16f1fe9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16ec06cde0 0x7f16ec148650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46436/0 (socket says 172.21.15.87:46436) 2024-09-17T03:16:05.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.288+0000 7f16f1fe9640 1 -- 172.21.15.87:0/4100641696 learned_addr learned my addr 172.21.15.87:0/4100641696 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:05.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.288+0000 7f16f17e8640 1 --2- 172.21.15.87:0/4100641696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16ec0d0c90 0x7f16ec14a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:05.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.288+0000 7f16f27ea640 1 --2- 172.21.15.87:0/4100641696 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f16ec15cb10 0x7f16ec1f6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:05.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.288+0000 7f16f1fe9640 1 -- 172.21.15.87:0/4100641696 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f16ec15cb10 msgr2=0x7f16ec1f6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:05.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.288+0000 7f16f1fe9640 1 --2- 172.21.15.87:0/4100641696 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f16ec15cb10 0x7f16ec1f6a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:05.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.288+0000 7f16f1fe9640 1 -- 172.21.15.87:0/4100641696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16ec0d0c90 msgr2=0x7f16ec14a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:05.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.288+0000 7f16f1fe9640 1 --2- 172.21.15.87:0/4100641696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16ec0d0c90 0x7f16ec14a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:05.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.288+0000 7f16f1fe9640 1 -- 172.21.15.87:0/4100641696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16ec0b0c70 con 0x7f16ec06cde0 2024-09-17T03:16:05.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.288+0000 7f16f17e8640 1 --2- 172.21.15.87:0/4100641696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16ec0d0c90 0x7f16ec14a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:16:05.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.288+0000 7f16f1fe9640 1 --2- 172.21.15.87:0/4100641696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16ec06cde0 0x7f16ec148650 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f16e802aa90 tx=0x7f16e8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:05.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.288+0000 7f16e27fc640 1 -- 172.21.15.87:0/4100641696 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e8005c90 con 0x7f16ec06cde0 2024-09-17T03:16:05.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.288+0000 7f16e27fc640 1 -- 172.21.15.87:0/4100641696 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16e8005020 con 0x7f16ec06cde0 2024-09-17T03:16:05.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.288+0000 7f16f424d640 1 -- 172.21.15.87:0/4100641696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16ec14b0b0 con 0x7f16ec06cde0 2024-09-17T03:16:05.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.288+0000 7f16e27fc640 1 -- 172.21.15.87:0/4100641696 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16e8005450 con 0x7f16ec06cde0 2024-09-17T03:16:05.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.288+0000 7f16f424d640 1 -- 172.21.15.87:0/4100641696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16ec14b670 con 0x7f16ec06cde0 2024-09-17T03:16:05.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.289+0000 7f16f424d640 1 -- 172.21.15.87:0/4100641696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16ec0ce5b0 con 0x7f16ec06cde0 2024-09-17T03:16:05.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.290+0000 7f16e27fc640 1 -- 172.21.15.87:0/4100641696 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f16e80317c0 con 0x7f16ec06cde0 2024-09-17T03:16:05.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.291+0000 7f16e27fc640 1 --2- 172.21.15.87:0/4100641696 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f16d0078bc0 0x7f16d007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:05.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.291+0000 7f16e27fc640 1 -- 172.21.15.87:0/4100641696 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5738+0+0 (secure 0 0 0) 0x7f16e80bbc90 con 0x7f16ec06cde0 2024-09-17T03:16:05.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.291+0000 7f16f17e8640 1 --2- 172.21.15.87:0/4100641696 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f16d0078bc0 0x7f16d007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:05.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.291+0000 7f16f17e8640 1 --2- 172.21.15.87:0/4100641696 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f16d0078bc0 0x7f16d007b0d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f16ec06a750 tx=0x7f16dc005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:05.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.295+0000 7f16e27fc640 1 -- 172.21.15.87:0/4100641696 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f16e803f030 con 0x7f16ec06cde0 2024-09-17T03:16:05.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.506+0000 7f16f424d640 1 -- 172.21.15.87:0/4100641696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f16ec06dac0 con 0x7f16ec06cde0 2024-09-17T03:16:05.505 INFO:tasks.ceph.mon.a.smithi087.stderr:2024-09-17T03:16:05.506+0000 7fafd01f9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-17T03:16:05.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.508+0000 7f16e27fc640 1 -- 172.21.15.87:0/4100641696 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) ==== 161+0+0 (secure 0 0 0) 0x7f16e8083a70 con 0x7f16ec06cde0 2024-09-17T03:16:05.506 INFO:teuthology.orchestra.run.smithi087.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-17T03:16:05.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.511+0000 7f16f424d640 1 -- 172.21.15.87:0/4100641696 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f16d0078bc0 msgr2=0x7f16d007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:05.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.511+0000 7f16f424d640 1 --2- 172.21.15.87:0/4100641696 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f16d0078bc0 0x7f16d007b0d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f16ec06a750 tx=0x7f16dc005f90 comp rx=0 tx=0).stop 2024-09-17T03:16:05.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.511+0000 7f16f424d640 1 -- 172.21.15.87:0/4100641696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16ec06cde0 msgr2=0x7f16ec148650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:05.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.511+0000 7f16f424d640 1 --2- 172.21.15.87:0/4100641696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16ec06cde0 0x7f16ec148650 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f16e802aa90 tx=0x7f16e8003c10 comp rx=0 tx=0).stop 2024-09-17T03:16:05.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.511+0000 7f16f424d640 1 -- 172.21.15.87:0/4100641696 shutdown_connections 2024-09-17T03:16:05.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.511+0000 7f16f424d640 1 --2- 172.21.15.87:0/4100641696 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f16d0078bc0 0x7f16d007b0d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:05.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.511+0000 7f16f424d640 1 --2- 172.21.15.87:0/4100641696 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f16ec15cb10 0x7f16ec1f6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:05.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.511+0000 7f16f424d640 1 --2- 172.21.15.87:0/4100641696 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16ec0d0c90 0x7f16ec14a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:05.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.511+0000 7f16f424d640 1 --2- 172.21.15.87:0/4100641696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16ec06cde0 0x7f16ec148650 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:05.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.511+0000 7f16f424d640 1 -- 172.21.15.87:0/4100641696 >> 172.21.15.87:0/4100641696 conn(0x7f16ec0d1180 msgr2=0x7f16ec15f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:05.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.511+0000 7f16f424d640 1 -- 172.21.15.87:0/4100641696 shutdown_connections 2024-09-17T03:16:05.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.511+0000 7f16f424d640 1 -- 172.21.15.87:0/4100641696 wait complete. 2024-09-17T03:16:05.511 INFO:tasks.ceph.mds.b.smithi149.stderr: -4037> 2024-09-17T03:15:34.057+0000 7f78a04ce640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-17T03:16:05.512 INFO:tasks.ceph.mds.b.smithi149.stderr: -2679> 2024-09-17T03:15:44.692+0000 7f789a4c2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-17T03:16:05.521 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-17T03:16:05.569 INFO:tasks.ceph.mds.b.smithi149.stdout:starting mds.b at 2024-09-17T03:16:05.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.638+0000 7f0660407640 1 -- 172.21.15.87:0/2822306057 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0658155d20 msgr2=0x7f06580d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:05.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.638+0000 7f0660407640 1 --2- 172.21.15.87:0/2822306057 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0658155d20 0x7f06580d0380 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f06580cec70 tx=0x7f064802aa30 comp rx=0 tx=0).stop 2024-09-17T03:16:05.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.638+0000 7f0660407640 1 -- 172.21.15.87:0/2822306057 shutdown_connections 2024-09-17T03:16:05.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.638+0000 7f0660407640 1 --2- 172.21.15.87:0/2822306057 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f06580742f0 0x7f065815f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:05.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.638+0000 7f0660407640 1 --2- 172.21.15.87:0/2822306057 >> v2:172.21.15.149:3300/0 conn(0x7f06580d08c0 0x7f0658073db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:05.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.638+0000 7f0660407640 1 --2- 172.21.15.87:0/2822306057 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0658155d20 0x7f06580d0380 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:05.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.638+0000 7f0660407640 1 -- 172.21.15.87:0/2822306057 >> 172.21.15.87:0/2822306057 conn(0x7f06580cdbc0 msgr2=0x7f06580cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:05.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.638+0000 7f0660407640 1 -- 172.21.15.87:0/2822306057 shutdown_connections 2024-09-17T03:16:05.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.638+0000 7f0660407640 1 -- 172.21.15.87:0/2822306057 wait complete. 2024-09-17T03:16:05.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.640+0000 7f0660407640 1 Processor -- start 2024-09-17T03:16:05.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.640+0000 7f0660407640 1 -- start start 2024-09-17T03:16:05.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.641+0000 7f0660407640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06580742f0 0x7f06580c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:05.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.641+0000 7f0660407640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f06580d08c0 0x7f06581f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:05.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.641+0000 7f0660407640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0658155d20 0x7f06580c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:05.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.641+0000 7f0660407640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f065815f6f0 con 0x7f0658155d20 2024-09-17T03:16:05.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.641+0000 7f0660407640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f065815f570 con 0x7f06580742f0 2024-09-17T03:16:05.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.641+0000 7f0660407640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f065815f870 con 0x7f06580d08c0 2024-09-17T03:16:05.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.642+0000 7f065d9a2640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f06580d08c0 0x7f06581f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:05.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.642+0000 7f065e9a4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0658155d20 0x7f06580c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:05.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.642+0000 7f065d9a2640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f06580d08c0 0x7f06581f4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.149:3301/0 says I am v2:172.21.15.87:57514/0 (socket says 172.21.15.87:57514) 2024-09-17T03:16:05.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.642+0000 7f065e9a4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0658155d20 0x7f06580c77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46446/0 (socket says 172.21.15.87:46446) 2024-09-17T03:16:05.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.642+0000 7f065d9a2640 1 -- 172.21.15.87:0/1981952985 learned_addr learned my addr 172.21.15.87:0/1981952985 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:05.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.642+0000 7f065e1a3640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06580742f0 0x7f06580c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:05.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.642+0000 7f065e9a4640 1 -- 172.21.15.87:0/1981952985 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f06580d08c0 msgr2=0x7f06581f4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:05.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.642+0000 7f065e9a4640 1 --2- 172.21.15.87:0/1981952985 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f06580d08c0 0x7f06581f4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:05.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.642+0000 7f065e9a4640 1 -- 172.21.15.87:0/1981952985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06580742f0 msgr2=0x7f06580c6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:05.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.642+0000 7f065e9a4640 1 --2- 172.21.15.87:0/1981952985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06580742f0 0x7f06580c6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:05.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.642+0000 7f065e9a4640 1 -- 172.21.15.87:0/1981952985 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06580c70c0 con 0x7f0658155d20 2024-09-17T03:16:05.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.642+0000 7f065e9a4640 1 --2- 172.21.15.87:0/1981952985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0658155d20 0x7f06580c77f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f065000e670 tx=0x7f065000eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:05.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.642+0000 7f0656ffd640 1 -- 172.21.15.87:0/1981952985 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650004220 con 0x7f0658155d20 2024-09-17T03:16:05.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.642+0000 7f0656ffd640 1 -- 172.21.15.87:0/1981952985 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f065000c8b0 con 0x7f0658155d20 2024-09-17T03:16:05.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.642+0000 7f0660407640 1 -- 172.21.15.87:0/1981952985 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06580c6b00 con 0x7f0658155d20 2024-09-17T03:16:05.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.642+0000 7f0656ffd640 1 -- 172.21.15.87:0/1981952985 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065000cd00 con 0x7f0658155d20 2024-09-17T03:16:05.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.642+0000 7f0660407640 1 -- 172.21.15.87:0/1981952985 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06580c99b0 con 0x7f0658155d20 2024-09-17T03:16:05.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.643+0000 7f0660407640 1 -- 172.21.15.87:0/1981952985 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f065815fd90 con 0x7f0658155d20 2024-09-17T03:16:05.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.644+0000 7f0656ffd640 1 -- 172.21.15.87:0/1981952985 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f06500035f0 con 0x7f0658155d20 2024-09-17T03:16:05.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.645+0000 7f0656ffd640 1 --2- 172.21.15.87:0/1981952985 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0644078bc0 0x7f064407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:05.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.645+0000 7f0656ffd640 1 -- 172.21.15.87:0/1981952985 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5824+0+0 (secure 0 0 0) 0x7f065009b140 con 0x7f0658155d20 2024-09-17T03:16:05.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.645+0000 7f065e1a3640 1 --2- 172.21.15.87:0/1981952985 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0644078bc0 0x7f064407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:05.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.645+0000 7f065e1a3640 1 --2- 172.21.15.87:0/1981952985 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0644078bc0 0x7f064407b0d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0648035000 tx=0x7f064803f040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:05.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.649+0000 7f0656ffd640 1 -- 172.21.15.87:0/1981952985 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0650062ed0 con 0x7f0658155d20 2024-09-17T03:16:05.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:05.860+0000 7f0660407640 1 -- 172.21.15.87:0/1981952985 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f06580c6890 con 0x7f0658155d20 2024-09-17T03:16:06.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.566+0000 7f0656ffd640 1 -- 172.21.15.87:0/1981952985 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) ==== 104+0+0 (secure 0 0 0) 0x7f0650067900 con 0x7f0658155d20 2024-09-17T03:16:06.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.568+0000 7f0660407640 1 -- 172.21.15.87:0/1981952985 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0644078bc0 msgr2=0x7f064407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:06.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.568+0000 7f0660407640 1 --2- 172.21.15.87:0/1981952985 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0644078bc0 0x7f064407b0d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f0648035000 tx=0x7f064803f040 comp rx=0 tx=0).stop 2024-09-17T03:16:06.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.568+0000 7f0660407640 1 -- 172.21.15.87:0/1981952985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0658155d20 msgr2=0x7f06580c77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:06.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.568+0000 7f0660407640 1 --2- 172.21.15.87:0/1981952985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0658155d20 0x7f06580c77f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f065000e670 tx=0x7f065000eb30 comp rx=0 tx=0).stop 2024-09-17T03:16:06.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.569+0000 7f0660407640 1 -- 172.21.15.87:0/1981952985 shutdown_connections 2024-09-17T03:16:06.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.569+0000 7f0660407640 1 --2- 172.21.15.87:0/1981952985 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0644078bc0 0x7f064407b0d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:06.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.569+0000 7f0660407640 1 --2- 172.21.15.87:0/1981952985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0658155d20 0x7f06580c77f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:06.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.569+0000 7f0660407640 1 --2- 172.21.15.87:0/1981952985 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f06580d08c0 0x7f06581f4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:06.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.569+0000 7f0660407640 1 --2- 172.21.15.87:0/1981952985 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f06580742f0 0x7f06580c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:06.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.569+0000 7f0660407640 1 -- 172.21.15.87:0/1981952985 >> 172.21.15.87:0/1981952985 conn(0x7f06580cdbc0 msgr2=0x7f06580cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:06.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.569+0000 7f0660407640 1 -- 172.21.15.87:0/1981952985 shutdown_connections 2024-09-17T03:16:06.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.569+0000 7f0660407640 1 -- 172.21.15.87:0/1981952985 wait complete. 2024-09-17T03:16:06.582 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-17T03:16:06.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.706+0000 7f01d1a8b640 1 -- 172.21.15.87:0/2312695596 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01b4003a00 msgr2=0x7f01b4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:06.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.706+0000 7f01d1a8b640 1 --2- 172.21.15.87:0/2312695596 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01b4003a00 0x7f01b4003e70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f01bc006fd0 tx=0x7f01b4005830 comp rx=0 tx=0).stop 2024-09-17T03:16:06.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.706+0000 7f01cbfff640 1 -- 172.21.15.87:0/2312695596 reap_dead start 2024-09-17T03:16:06.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.706+0000 7f01d1a8b640 1 -- 172.21.15.87:0/2312695596 shutdown_connections 2024-09-17T03:16:06.705 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.706+0000 7f01d1a8b640 1 -- 172.21.15.87:0/2312695596 >> 172.21.15.87:0/2312695596 conn(0x7f01cc0d1180 msgr2=0x7f01cc0d1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:06.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.706+0000 7f01d1a8b640 1 -- 172.21.15.87:0/2312695596 shutdown_connections 2024-09-17T03:16:06.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.706+0000 7f01d1a8b640 1 -- 172.21.15.87:0/2312695596 wait complete. 2024-09-17T03:16:06.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.707+0000 7f01d1a8b640 1 Processor -- start 2024-09-17T03:16:06.706 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.708+0000 7f01d1a8b640 1 -- start start 2024-09-17T03:16:06.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.709+0000 7f01d1a8b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01b4003a00 0x7f01cc150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:06.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.709+0000 7f01d1a8b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01cc14c110 0x7f01cc0acb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:06.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.709+0000 7f01d1a8b640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01cc0ad0b0 0x7f01cc1f5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:06.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.709+0000 7f01d1a8b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc15c8e0 con 0x7f01cc14c110 2024-09-17T03:16:06.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.709+0000 7f01d1a8b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc15c760 con 0x7f01b4003a00 2024-09-17T03:16:06.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.709+0000 7f01d1a8b640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f01cc15ca60 con 0x7f01cc0ad0b0 2024-09-17T03:16:06.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.709+0000 7f01caffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01cc14c110 0x7f01cc0acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:06.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.709+0000 7f01caffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01cc14c110 0x7f01cc0acb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46466/0 (socket says 172.21.15.87:46466) 2024-09-17T03:16:06.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.709+0000 7f01caffd640 1 -- 172.21.15.87:0/3134481673 learned_addr learned my addr 172.21.15.87:0/3134481673 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:06.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.709+0000 7f01cb7fe640 1 --2- 172.21.15.87:0/3134481673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01b4003a00 0x7f01cc150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:06.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.709+0000 7f01cbfff640 1 --2- 172.21.15.87:0/3134481673 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01cc0ad0b0 0x7f01cc1f5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:06.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.709+0000 7f01caffd640 1 -- 172.21.15.87:0/3134481673 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01cc0ad0b0 msgr2=0x7f01cc1f5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:06.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.709+0000 7f01caffd640 1 --2- 172.21.15.87:0/3134481673 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01cc0ad0b0 0x7f01cc1f5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:06.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.709+0000 7f01caffd640 1 -- 172.21.15.87:0/3134481673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01b4003a00 msgr2=0x7f01cc150160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:06.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.709+0000 7f01caffd640 1 --2- 172.21.15.87:0/3134481673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01b4003a00 0x7f01cc150160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:06.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.709+0000 7f01caffd640 1 -- 172.21.15.87:0/3134481673 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01cc1e2680 con 0x7f01cc14c110 2024-09-17T03:16:06.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.709+0000 7f01cbfff640 1 --2- 172.21.15.87:0/3134481673 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01cc0ad0b0 0x7f01cc1f5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:16:06.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.709+0000 7f01caffd640 1 --2- 172.21.15.87:0/3134481673 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01cc14c110 0x7f01cc0acb70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f01c0011780 tx=0x7f01c0011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:06.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.710+0000 7f01b3fff640 1 -- 172.21.15.87:0/3134481673 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c0004460 con 0x7f01cc14c110 2024-09-17T03:16:06.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.710+0000 7f01b3fff640 1 -- 172.21.15.87:0/3134481673 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01c0016070 con 0x7f01cc14c110 2024-09-17T03:16:06.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.710+0000 7f01d1a8b640 1 -- 172.21.15.87:0/3134481673 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01cc1e2960 con 0x7f01cc14c110 2024-09-17T03:16:06.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.710+0000 7f01b3fff640 1 -- 172.21.15.87:0/3134481673 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c000fba0 con 0x7f01cc14c110 2024-09-17T03:16:06.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.710+0000 7f01d1a8b640 1 -- 172.21.15.87:0/3134481673 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01cc1e2f20 con 0x7f01cc14c110 2024-09-17T03:16:06.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.711+0000 7f01d1a8b640 1 -- 172.21.15.87:0/3134481673 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01cc1f6650 con 0x7f01cc14c110 2024-09-17T03:16:06.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.712+0000 7f01b3fff640 1 -- 172.21.15.87:0/3134481673 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f01c0018030 con 0x7f01cc14c110 2024-09-17T03:16:06.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.712+0000 7f01b3fff640 1 --2- 172.21.15.87:0/3134481673 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f01b8078b90 0x7f01b807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:06.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.712+0000 7f01b3fff640 1 -- 172.21.15.87:0/3134481673 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5824+0+0 (secure 0 0 0) 0x7f01c009b3b0 con 0x7f01cc14c110 2024-09-17T03:16:06.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.712+0000 7f01cb7fe640 1 --2- 172.21.15.87:0/3134481673 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f01b8078b90 0x7f01b807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:06.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.713+0000 7f01cb7fe640 1 --2- 172.21.15.87:0/3134481673 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f01b8078b90 0x7f01b807b0a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f01bc033000 tx=0x7f01bc001360 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:06.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.716+0000 7f01b3fff640 1 -- 172.21.15.87:0/3134481673 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f01c00638a0 con 0x7f01cc14c110 2024-09-17T03:16:06.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:06.891+0000 7f01d1a8b640 1 -- 172.21.15.87:0/3134481673 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f01cc15c220 con 0x7f01cc14c110 2024-09-17T03:16:07.578 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.578+0000 7f01b3fff640 1 -- 172.21.15.87:0/3134481673 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v54) ==== 181+0+0 (secure 0 0 0) 0x7f01c00631a0 con 0x7f01cc14c110 2024-09-17T03:16:07.579 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' removed 2024-09-17T03:16:07.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.581+0000 7f01d1a8b640 1 -- 172.21.15.87:0/3134481673 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f01b8078b90 msgr2=0x7f01b807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:07.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.581+0000 7f01d1a8b640 1 --2- 172.21.15.87:0/3134481673 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f01b8078b90 0x7f01b807b0a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f01bc033000 tx=0x7f01bc001360 comp rx=0 tx=0).stop 2024-09-17T03:16:07.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.581+0000 7f01d1a8b640 1 -- 172.21.15.87:0/3134481673 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01cc14c110 msgr2=0x7f01cc0acb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:07.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.581+0000 7f01d1a8b640 1 --2- 172.21.15.87:0/3134481673 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01cc14c110 0x7f01cc0acb70 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f01c0011780 tx=0x7f01c0011c40 comp rx=0 tx=0).stop 2024-09-17T03:16:07.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.581+0000 7f01d1a8b640 1 -- 172.21.15.87:0/3134481673 shutdown_connections 2024-09-17T03:16:07.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.581+0000 7f01d1a8b640 1 --2- 172.21.15.87:0/3134481673 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f01b8078b90 0x7f01b807b0a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:07.581 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.581+0000 7f01d1a8b640 1 --2- 172.21.15.87:0/3134481673 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f01cc0ad0b0 0x7f01cc1f5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:07.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.581+0000 7f01d1a8b640 1 --2- 172.21.15.87:0/3134481673 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01cc14c110 0x7f01cc0acb70 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:07.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.581+0000 7f01d1a8b640 1 --2- 172.21.15.87:0/3134481673 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f01b4003a00 0x7f01cc150160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:07.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.581+0000 7f01d1a8b640 1 -- 172.21.15.87:0/3134481673 >> 172.21.15.87:0/3134481673 conn(0x7f01cc0d1180 msgr2=0x7f01cc0b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:07.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.581+0000 7f01d1a8b640 1 -- 172.21.15.87:0/3134481673 shutdown_connections 2024-09-17T03:16:07.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.581+0000 7f01d1a8b640 1 -- 172.21.15.87:0/3134481673 wait complete. 2024-09-17T03:16:07.593 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-17T03:16:07.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.711+0000 7f7cccf20640 1 -- 172.21.15.87:0/2636935502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc8161dd0 msgr2=0x7f7cc80cf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:07.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.711+0000 7f7cccf20640 1 --2- 172.21.15.87:0/2636935502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc8161dd0 0x7f7cc80cf760 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc009830 tx=0x7f7cbc02a510 comp rx=0 tx=0).stop 2024-09-17T03:16:07.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.711+0000 7f7cccf20640 1 -- 172.21.15.87:0/2636935502 shutdown_connections 2024-09-17T03:16:07.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.711+0000 7f7cccf20640 1 --2- 172.21.15.87:0/2636935502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc8161dd0 0x7f7cc80cf760 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:07.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.711+0000 7f7cccf20640 1 --2- 172.21.15.87:0/2636935502 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7cc80b3ae0 0x7f7cc80b95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:07.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.711+0000 7f7cccf20640 1 -- 172.21.15.87:0/2636935502 >> 172.21.15.87:0/2636935502 conn(0x7f7cc809d7e0 msgr2=0x7f7cc808bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:07.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.711+0000 7f7cccf20640 1 -- 172.21.15.87:0/2636935502 shutdown_connections 2024-09-17T03:16:07.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.711+0000 7f7cccf20640 1 -- 172.21.15.87:0/2636935502 wait complete. 2024-09-17T03:16:07.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.713+0000 7f7cccf20640 1 Processor -- start 2024-09-17T03:16:07.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.713+0000 7f7cccf20640 1 -- start start 2024-09-17T03:16:07.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.714+0000 7f7cccf20640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cc80b3ae0 0x7f7cc81f9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:07.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.714+0000 7f7cccf20640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc80d0d00 0x7f7cc815b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:07.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.714+0000 7f7cccf20640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7cc8161dd0 0x7f7cc8160b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:07.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.714+0000 7f7cccf20640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc80d2750 con 0x7f7cc80d0d00 2024-09-17T03:16:07.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.714+0000 7f7cccf20640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc80d25d0 con 0x7f7cc80b3ae0 2024-09-17T03:16:07.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.714+0000 7f7cccf20640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f7cc80d28d0 con 0x7f7cc8161dd0 2024-09-17T03:16:07.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.715+0000 7f7cc5d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc80d0d00 0x7f7cc815b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:07.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.715+0000 7f7cc5d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc80d0d00 0x7f7cc815b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46476/0 (socket says 172.21.15.87:46476) 2024-09-17T03:16:07.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.715+0000 7f7cc5d9b640 1 -- 172.21.15.87:0/2710450477 learned_addr learned my addr 172.21.15.87:0/2710450477 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:07.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.715+0000 7f7cc6d9d640 1 --2- 172.21.15.87:0/2710450477 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7cc8161dd0 0x7f7cc8160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:07.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.715+0000 7f7cc659c640 1 --2- 172.21.15.87:0/2710450477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cc80b3ae0 0x7f7cc81f9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:07.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.715+0000 7f7cc5d9b640 1 -- 172.21.15.87:0/2710450477 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7cc8161dd0 msgr2=0x7f7cc8160b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:07.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.715+0000 7f7cc5d9b640 1 --2- 172.21.15.87:0/2710450477 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7cc8161dd0 0x7f7cc8160b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:07.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.715+0000 7f7cc5d9b640 1 -- 172.21.15.87:0/2710450477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cc80b3ae0 msgr2=0x7f7cc81f9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:07.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.715+0000 7f7cc5d9b640 1 --2- 172.21.15.87:0/2710450477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cc80b3ae0 0x7f7cc81f9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:07.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.715+0000 7f7cc5d9b640 1 -- 172.21.15.87:0/2710450477 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc81f5530 con 0x7f7cc80d0d00 2024-09-17T03:16:07.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.715+0000 7f7cc659c640 1 --2- 172.21.15.87:0/2710450477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cc80b3ae0 0x7f7cc81f9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:16:07.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.715+0000 7f7cc5d9b640 1 --2- 172.21.15.87:0/2710450477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc80d0d00 0x7f7cc815b720 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc02ab00 tx=0x7f7cbc033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:07.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.715+0000 7f7cb6ffd640 1 -- 172.21.15.87:0/2710450477 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc031d60 con 0x7f7cc80d0d00 2024-09-17T03:16:07.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.715+0000 7f7cb6ffd640 1 -- 172.21.15.87:0/2710450477 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cbc033e10 con 0x7f7cc80d0d00 2024-09-17T03:16:07.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.715+0000 7f7cb6ffd640 1 -- 172.21.15.87:0/2710450477 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc005260 con 0x7f7cc80d0d00 2024-09-17T03:16:07.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.715+0000 7f7cccf20640 1 -- 172.21.15.87:0/2710450477 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc81f5810 con 0x7f7cc80d0d00 2024-09-17T03:16:07.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.715+0000 7f7cccf20640 1 -- 172.21.15.87:0/2710450477 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc81f5dd0 con 0x7f7cc80d0d00 2024-09-17T03:16:07.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.716+0000 7f7cccf20640 1 -- 172.21.15.87:0/2710450477 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cc80d2dc0 con 0x7f7cc80d0d00 2024-09-17T03:16:07.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.717+0000 7f7cb6ffd640 1 -- 172.21.15.87:0/2710450477 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7cbc005400 con 0x7f7cc80d0d00 2024-09-17T03:16:07.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.718+0000 7f7cb6ffd640 1 --2- 172.21.15.87:0/2710450477 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7ca8078b90 0x7f7ca807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:07.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.718+0000 7f7cb6ffd640 1 -- 172.21.15.87:0/2710450477 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5386+0+0 (secure 0 0 0) 0x7f7cbc0bbe50 con 0x7f7cc80d0d00 2024-09-17T03:16:07.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.718+0000 7f7cc659c640 1 --2- 172.21.15.87:0/2710450477 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7ca8078b90 0x7f7ca807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:07.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.719+0000 7f7cc659c640 1 --2- 172.21.15.87:0/2710450477 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7ca8078b90 0x7f7ca807b0a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800ab40 tx=0x7f7cb800a520 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:07.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.722+0000 7f7cb6ffd640 1 -- 172.21.15.87:0/2710450477 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f7cbc03f030 con 0x7f7cc80d0d00 2024-09-17T03:16:07.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:07.896+0000 7f7cccf20640 1 -- 172.21.15.87:0/2710450477 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7cc80caea0 con 0x7f7cc80d0d00 2024-09-17T03:16:08.584 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.584+0000 7f7cb6ffd640 1 -- 172.21.15.87:0/2710450477 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v55) ==== 169+0+0 (secure 0 0 0) 0x7f7cbc083d80 con 0x7f7cc80d0d00 2024-09-17T03:16:08.585 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' removed 2024-09-17T03:16:08.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.587+0000 7f7cccf20640 1 -- 172.21.15.87:0/2710450477 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7ca8078b90 msgr2=0x7f7ca807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:08.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.587+0000 7f7cccf20640 1 --2- 172.21.15.87:0/2710450477 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7ca8078b90 0x7f7ca807b0a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800ab40 tx=0x7f7cb800a520 comp rx=0 tx=0).stop 2024-09-17T03:16:08.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.587+0000 7f7cccf20640 1 -- 172.21.15.87:0/2710450477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc80d0d00 msgr2=0x7f7cc815b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:08.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.587+0000 7f7cccf20640 1 --2- 172.21.15.87:0/2710450477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc80d0d00 0x7f7cc815b720 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc02ab00 tx=0x7f7cbc033c20 comp rx=0 tx=0).stop 2024-09-17T03:16:08.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.587+0000 7f7cccf20640 1 -- 172.21.15.87:0/2710450477 shutdown_connections 2024-09-17T03:16:08.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.587+0000 7f7cccf20640 1 --2- 172.21.15.87:0/2710450477 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f7ca8078b90 0x7f7ca807b0a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:08.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.587+0000 7f7cccf20640 1 --2- 172.21.15.87:0/2710450477 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f7cc8161dd0 0x7f7cc8160b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:08.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.587+0000 7f7cccf20640 1 --2- 172.21.15.87:0/2710450477 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7cc80d0d00 0x7f7cc815b720 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:08.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.587+0000 7f7cccf20640 1 --2- 172.21.15.87:0/2710450477 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f7cc80b3ae0 0x7f7cc81f9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:08.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.587+0000 7f7cccf20640 1 -- 172.21.15.87:0/2710450477 >> 172.21.15.87:0/2710450477 conn(0x7f7cc809d7e0 msgr2=0x7f7cc81eaa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:08.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.587+0000 7f7cccf20640 1 -- 172.21.15.87:0/2710450477 shutdown_connections 2024-09-17T03:16:08.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.587+0000 7f7cccf20640 1 -- 172.21.15.87:0/2710450477 wait complete. 2024-09-17T03:16:08.599 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-17T03:16:08.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.715+0000 7f166b60e640 1 -- 172.21.15.87:0/1875099989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16640d0930 msgr2=0x7f16640cf250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:08.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.715+0000 7f166b60e640 1 --2- 172.21.15.87:0/1875099989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16640d0930 0x7f16640cf250 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f16640b4e00 tx=0x7f165c02a980 comp rx=0 tx=0).stop 2024-09-17T03:16:08.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.715+0000 7f166b60e640 1 -- 172.21.15.87:0/1875099989 shutdown_connections 2024-09-17T03:16:08.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.715+0000 7f166b60e640 1 --2- 172.21.15.87:0/1875099989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16640d0930 0x7f16640cf250 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:08.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.715+0000 7f166b60e640 1 --2- 172.21.15.87:0/1875099989 >> v2:172.21.15.149:3300/0 conn(0x7f16640b7860 0x7f16640b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:08.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.715+0000 7f166b60e640 1 -- 172.21.15.87:0/1875099989 >> 172.21.15.87:0/1875099989 conn(0x7f16640d2230 msgr2=0x7f16640d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:08.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.715+0000 7f166b60e640 1 -- 172.21.15.87:0/1875099989 shutdown_connections 2024-09-17T03:16:08.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.715+0000 7f166b60e640 1 -- 172.21.15.87:0/1875099989 wait complete. 2024-09-17T03:16:08.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.717+0000 7f166b60e640 1 Processor -- start 2024-09-17T03:16:08.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.717+0000 7f166b60e640 1 -- start start 2024-09-17T03:16:08.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.718+0000 7f166b60e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16640b7860 0x7f16641f8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:08.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.718+0000 7f166b60e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16640d0520 0x7f166415f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:08.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.718+0000 7f166b60e640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f16640d0930 0x7f16641f8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:08.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.718+0000 7f166b60e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f16640cf910 con 0x7f16640d0520 2024-09-17T03:16:08.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.718+0000 7f166b60e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f16640cf790 con 0x7f16640b7860 2024-09-17T03:16:08.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.718+0000 7f166b60e640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f1664168f20 con 0x7f16640d0930 2024-09-17T03:16:08.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.718+0000 7f1668ba9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16640d0520 0x7f166415f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:08.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.719+0000 7f1668ba9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16640d0520 0x7f166415f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46484/0 (socket says 172.21.15.87:46484) 2024-09-17T03:16:08.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.719+0000 7f1668ba9640 1 -- 172.21.15.87:0/3774790393 learned_addr learned my addr 172.21.15.87:0/3774790393 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:08.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.719+0000 7f16693aa640 1 --2- 172.21.15.87:0/3774790393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16640b7860 0x7f16641f8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:08.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.719+0000 7f1669bab640 1 --2- 172.21.15.87:0/3774790393 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f16640d0930 0x7f16641f8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:08.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.719+0000 7f1668ba9640 1 -- 172.21.15.87:0/3774790393 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f16640d0930 msgr2=0x7f16641f8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:08.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.719+0000 7f1668ba9640 1 --2- 172.21.15.87:0/3774790393 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f16640d0930 0x7f16641f8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:08.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.719+0000 7f1668ba9640 1 -- 172.21.15.87:0/3774790393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16640b7860 msgr2=0x7f16641f8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:08.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.719+0000 7f1668ba9640 1 --2- 172.21.15.87:0/3774790393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16640b7860 0x7f16641f8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:08.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.719+0000 7f1668ba9640 1 -- 172.21.15.87:0/3774790393 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16641f9a10 con 0x7f16640d0520 2024-09-17T03:16:08.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.719+0000 7f1668ba9640 1 --2- 172.21.15.87:0/3774790393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16640d0520 0x7f166415f140 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f165400b410 tx=0x7f165400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:08.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.719+0000 7f1659ffb640 1 -- 172.21.15.87:0/3774790393 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654015020 con 0x7f16640d0520 2024-09-17T03:16:08.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.719+0000 7f1659ffb640 1 -- 172.21.15.87:0/3774790393 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1654011920 con 0x7f16640d0520 2024-09-17T03:16:08.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.719+0000 7f166b60e640 1 -- 172.21.15.87:0/3774790393 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1664163830 con 0x7f16640d0520 2024-09-17T03:16:08.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.719+0000 7f1659ffb640 1 -- 172.21.15.87:0/3774790393 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654011d70 con 0x7f16640d0520 2024-09-17T03:16:08.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.719+0000 7f166b60e640 1 -- 172.21.15.87:0/3774790393 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1664163df0 con 0x7f16640d0520 2024-09-17T03:16:08.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.720+0000 7f166b60e640 1 -- 172.21.15.87:0/3774790393 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f166415fa90 con 0x7f16640d0520 2024-09-17T03:16:08.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.721+0000 7f1659ffb640 1 -- 172.21.15.87:0/3774790393 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1654018020 con 0x7f16640d0520 2024-09-17T03:16:08.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.721+0000 7f1659ffb640 1 --2- 172.21.15.87:0/3774790393 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1648078b90 0x7f164807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:08.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.722+0000 7f1659ffb640 1 -- 172.21.15.87:0/3774790393 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f165409a750 con 0x7f16640d0520 2024-09-17T03:16:08.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.722+0000 7f16693aa640 1 --2- 172.21.15.87:0/3774790393 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1648078b90 0x7f164807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:08.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.722+0000 7f16693aa640 1 --2- 172.21.15.87:0/3774790393 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1648078b90 0x7f164807b0a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f16600098e0 tx=0x7f1660009060 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:08.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.726+0000 7f1659ffb640 1 -- 172.21.15.87:0/3774790393 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f165401f020 con 0x7f16640d0520 2024-09-17T03:16:08.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.900+0000 7f166b60e640 1 -- 172.21.15.87:0/3774790393 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f16640cd670 con 0x7f16640d0520 2024-09-17T03:16:08.903 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.904+0000 7f1659ffb640 1 -- 172.21.15.87:0/3774790393 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v35) ==== 71+0+0 (secure 0 0 0) 0x7f1654062960 con 0x7f16640d0520 2024-09-17T03:16:08.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.906+0000 7f166b60e640 1 -- 172.21.15.87:0/3774790393 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1648078b90 msgr2=0x7f164807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:08.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.906+0000 7f166b60e640 1 --2- 172.21.15.87:0/3774790393 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1648078b90 0x7f164807b0a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f16600098e0 tx=0x7f1660009060 comp rx=0 tx=0).stop 2024-09-17T03:16:08.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.906+0000 7f166b60e640 1 -- 172.21.15.87:0/3774790393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16640d0520 msgr2=0x7f166415f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:08.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.906+0000 7f166b60e640 1 --2- 172.21.15.87:0/3774790393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16640d0520 0x7f166415f140 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f165400b410 tx=0x7f165400b8d0 comp rx=0 tx=0).stop 2024-09-17T03:16:08.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.906+0000 7f166b60e640 1 -- 172.21.15.87:0/3774790393 shutdown_connections 2024-09-17T03:16:08.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.906+0000 7f166b60e640 1 --2- 172.21.15.87:0/3774790393 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f1648078b90 0x7f164807b0a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:08.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.906+0000 7f166b60e640 1 --2- 172.21.15.87:0/3774790393 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f16640d0930 0x7f16641f8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:08.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.906+0000 7f166b60e640 1 --2- 172.21.15.87:0/3774790393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16640d0520 0x7f166415f140 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:08.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.906+0000 7f166b60e640 1 --2- 172.21.15.87:0/3774790393 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f16640b7860 0x7f16641f8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:08.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.906+0000 7f166b60e640 1 -- 172.21.15.87:0/3774790393 >> 172.21.15.87:0/3774790393 conn(0x7f16640d2230 msgr2=0x7f1664071dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:08.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.907+0000 7f166b60e640 1 -- 172.21.15.87:0/3774790393 shutdown_connections 2024-09-17T03:16:08.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:08.907+0000 7f166b60e640 1 -- 172.21.15.87:0/3774790393 wait complete. 2024-09-17T03:16:08.917 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm' 2024-09-17T03:16:09.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.030+0000 7f0418084640 1 -- 172.21.15.87:0/2136252452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04100ce0a0 msgr2=0x7f04100d0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:09.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.030+0000 7f0418084640 1 --2- 172.21.15.87:0/2136252452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04100ce0a0 0x7f04100d0020 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f040c009bb0 tx=0x7f040c02a820 comp rx=0 tx=0).stop 2024-09-17T03:16:09.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.030+0000 7f0418084640 1 -- 172.21.15.87:0/2136252452 shutdown_connections 2024-09-17T03:16:09.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.030+0000 7f0418084640 1 --2- 172.21.15.87:0/2136252452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04100ce0a0 0x7f04100d0020 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:09.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.030+0000 7f0418084640 1 -- 172.21.15.87:0/2136252452 >> 172.21.15.87:0/2136252452 conn(0x7f04100b3d30 msgr2=0x7f04100b4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:09.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.030+0000 7f0418084640 1 -- 172.21.15.87:0/2136252452 shutdown_connections 2024-09-17T03:16:09.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.030+0000 7f0418084640 1 -- 172.21.15.87:0/2136252452 wait complete. 2024-09-17T03:16:09.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.032+0000 7f0418084640 1 Processor -- start 2024-09-17T03:16:09.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.032+0000 7f0418084640 1 -- start start 2024-09-17T03:16:09.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.033+0000 7f0418084640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f041009aa30 0x7f0410149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:09.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.033+0000 7f0418084640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04100b5090 0x7f04101f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:09.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.033+0000 7f0418084640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04100ce0a0 0x7f04101566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:09.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.033+0000 7f0418084640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f04100cd0a0 con 0x7f04100b5090 2024-09-17T03:16:09.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.033+0000 7f0418084640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f041009b980 con 0x7f04100ce0a0 2024-09-17T03:16:09.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.033+0000 7f0418084640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f04100b5bb0 con 0x7f041009aa30 2024-09-17T03:16:09.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.033+0000 7f041561f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04100b5090 0x7f04101f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:09.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.033+0000 7f041561f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04100b5090 0x7f04101f43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46506/0 (socket says 172.21.15.87:46506) 2024-09-17T03:16:09.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.033+0000 7f041561f640 1 -- 172.21.15.87:0/3661277615 learned_addr learned my addr 172.21.15.87:0/3661277615 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:09.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.033+0000 7f0415e20640 1 --2- 172.21.15.87:0/3661277615 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f041009aa30 0x7f0410149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:09.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.033+0000 7f0416621640 1 --2- 172.21.15.87:0/3661277615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04100ce0a0 0x7f04101566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:09.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.034+0000 7f041561f640 1 -- 172.21.15.87:0/3661277615 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f041009aa30 msgr2=0x7f0410149750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:09.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.034+0000 7f041561f640 1 --2- 172.21.15.87:0/3661277615 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f041009aa30 0x7f0410149750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:09.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.034+0000 7f041561f640 1 -- 172.21.15.87:0/3661277615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04100ce0a0 msgr2=0x7f04101566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:09.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.034+0000 7f041561f640 1 --2- 172.21.15.87:0/3661277615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04100ce0a0 0x7f04101566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:09.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.034+0000 7f041561f640 1 -- 172.21.15.87:0/3661277615 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0410155fa0 con 0x7f04100b5090 2024-09-17T03:16:09.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.034+0000 7f041561f640 1 --2- 172.21.15.87:0/3661277615 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04100b5090 0x7f04101f43f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f040000c3a0 tx=0x7f040000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:09.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.034+0000 7f04067fc640 1 -- 172.21.15.87:0/3661277615 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0400011790 con 0x7f04100b5090 2024-09-17T03:16:09.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.034+0000 7f04067fc640 1 -- 172.21.15.87:0/3661277615 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0400017070 con 0x7f04100b5090 2024-09-17T03:16:09.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.034+0000 7f04067fc640 1 -- 172.21.15.87:0/3661277615 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0400011cc0 con 0x7f04100b5090 2024-09-17T03:16:09.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.034+0000 7f0418084640 1 -- 172.21.15.87:0/3661277615 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04101f4930 con 0x7f04100b5090 2024-09-17T03:16:09.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.034+0000 7f0418084640 1 -- 172.21.15.87:0/3661277615 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04100c6070 con 0x7f04100b5090 2024-09-17T03:16:09.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.035+0000 7f0418084640 1 -- 172.21.15.87:0/3661277615 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04100c5b00 con 0x7f04100b5090 2024-09-17T03:16:09.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.036+0000 7f04067fc640 1 -- 172.21.15.87:0/3661277615 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f040001c460 con 0x7f04100b5090 2024-09-17T03:16:09.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.036+0000 7f04067fc640 1 --2- 172.21.15.87:0/3661277615 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f03f4078b90 0x7f03f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:09.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.036+0000 7f04067fc640 1 -- 172.21.15.87:0/3661277615 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f040009a800 con 0x7f04100b5090 2024-09-17T03:16:09.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.036+0000 7f0415e20640 1 --2- 172.21.15.87:0/3661277615 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f03f4078b90 0x7f03f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:09.038 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.037+0000 7f0415e20640 1 --2- 172.21.15.87:0/3661277615 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f03f4078b90 0x7f03f407b0a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f04100cca20 tx=0x7f040c004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:09.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.041+0000 7f04067fc640 1 -- 172.21.15.87:0/3661277615 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f04101f4930 con 0x7f04100b5090 2024-09-17T03:16:09.140 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:16:09.143 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:16:09.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.298+0000 7f0418084640 1 -- 172.21.15.87:0/3661277615 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]} v 0) -- 0x7f041009b2e0 con 0x7f04100b5090 2024-09-17T03:16:09.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.586+0000 7f04067fc640 1 -- 172.21.15.87:0/3661277615 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_pool_perm.TestPoolPerm.test_pool_perm"]}]=0 v125) ==== 134+0+0 (secure 0 0 0) 0x7f041009b2e0 con 0x7f04100b5090 2024-09-17T03:16:09.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.588+0000 7f0418084640 1 -- 172.21.15.87:0/3661277615 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f03f4078b90 msgr2=0x7f03f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:09.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.588+0000 7f0418084640 1 --2- 172.21.15.87:0/3661277615 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f03f4078b90 0x7f03f407b0a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f04100cca20 tx=0x7f040c004ab0 comp rx=0 tx=0).stop 2024-09-17T03:16:09.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.588+0000 7f0418084640 1 -- 172.21.15.87:0/3661277615 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04100b5090 msgr2=0x7f04101f43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:09.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.588+0000 7f0418084640 1 --2- 172.21.15.87:0/3661277615 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04100b5090 0x7f04101f43f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f040000c3a0 tx=0x7f040000c860 comp rx=0 tx=0).stop 2024-09-17T03:16:09.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.588+0000 7f0418084640 1 -- 172.21.15.87:0/3661277615 shutdown_connections 2024-09-17T03:16:09.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.588+0000 7f0418084640 1 --2- 172.21.15.87:0/3661277615 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f03f4078b90 0x7f03f407b0a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:09.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.588+0000 7f0418084640 1 --2- 172.21.15.87:0/3661277615 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04100ce0a0 0x7f04101566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:09.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.588+0000 7f0418084640 1 --2- 172.21.15.87:0/3661277615 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04100b5090 0x7f04101f43f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:09.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.588+0000 7f0418084640 1 --2- 172.21.15.87:0/3661277615 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f041009aa30 0x7f0410149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:09.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.588+0000 7f0418084640 1 -- 172.21.15.87:0/3661277615 >> 172.21.15.87:0/3661277615 conn(0x7f04100b3d30 msgr2=0x7f0410070ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:09.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.588+0000 7f0418084640 1 -- 172.21.15.87:0/3661277615 shutdown_connections 2024-09-17T03:16:09.589 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:09.589+0000 7f0418084640 1 -- 172.21.15.87:0/3661277615 wait complete. 2024-09-17T03:16:09.601 INFO:tasks.cephfs_test_runner:test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) ... ERROR 2024-09-17T03:16:09.602 INFO:tasks.cephfs_test_runner: 2024-09-17T03:16:09.602 INFO:tasks.cephfs_test_runner:====================================================================== 2024-09-17T03:16:09.602 INFO:tasks.cephfs_test_runner:ERROR: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-09-17T03:16:09.602 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-17T03:16:09.602 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-09-17T03:16:09.602 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa/tasks/cephfs/test_pool_perm.py", line 52, in test_pool_perm 2024-09-17T03:16:09.602 INFO:tasks.cephfs_test_runner: self.mount_a.run_python(remote_script.format(path=file_path, check_read=str(True))) 2024-09-17T03:16:09.602 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa/tasks/cephfs/mount.py", line 766, in run_python 2024-09-17T03:16:09.602 INFO:tasks.cephfs_test_runner: p.wait() 2024-09-17T03:16:09.602 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait 2024-09-17T03:16:09.602 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-09-17T03:16:09.602 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-09-17T03:16:09.603 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-09-17T03:16:09.603 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi087 with status 1: 'stdin-killer -- python3 -c \'\nimport os\nimport errno\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR)\ntry:\n if True:\n ret = os.read(fd, 1024)\n else:\n os.write(fd, b\'"\'"\'content\'"\'"\')\nexcept OSError as e:\n if e.errno != errno.EPERM:\n raise\nelse:\n raise RuntimeError("client does not check permission of data pool")\n\'' 2024-09-17T03:16:09.603 INFO:tasks.cephfs_test_runner: 2024-09-17T03:16:09.603 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-17T03:16:09.603 INFO:tasks.cephfs_test_runner:Ran 2 tests in 91.850s 2024-09-17T03:16:09.603 INFO:tasks.cephfs_test_runner: 2024-09-17T03:16:09.603 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-09-17T03:16:09.603 INFO:tasks.cephfs_test_runner: 2024-09-17T03:16:09.603 INFO:tasks.cephfs_test_runner:====================================================================== 2024-09-17T03:16:09.603 INFO:tasks.cephfs_test_runner:ERROR: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-09-17T03:16:09.603 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-17T03:16:09.604 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-09-17T03:16:09.604 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa/tasks/cephfs/test_pool_perm.py", line 52, in test_pool_perm 2024-09-17T03:16:09.604 INFO:tasks.cephfs_test_runner: self.mount_a.run_python(remote_script.format(path=file_path, check_read=str(True))) 2024-09-17T03:16:09.604 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa/tasks/cephfs/mount.py", line 766, in run_python 2024-09-17T03:16:09.604 INFO:tasks.cephfs_test_runner: p.wait() 2024-09-17T03:16:09.604 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait 2024-09-17T03:16:09.604 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-09-17T03:16:09.604 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-09-17T03:16:09.604 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-09-17T03:16:09.604 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi087 with status 1: 'stdin-killer -- python3 -c \'\nimport os\nimport errno\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/test_file", os.O_RDWR)\ntry:\n if True:\n ret = os.read(fd, 1024)\n else:\n os.write(fd, b\'"\'"\'content\'"\'"\')\nexcept OSError as e:\n if e.errno != errno.EPERM:\n raise\nelse:\n raise RuntimeError("client does not check permission of data pool")\n\'' 2024-09-17T03:16:09.604 INFO:tasks.cephfs_test_runner: 2024-09-17T03:16:09.604 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 112, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-09-17T03:16:09.787 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=18421a23d5bc45bf8002ca7e50a51029 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 112, in run_tasks manager.__enter__() File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_ce1b342bf8c6349a8c90cd607f83bff2443f21cf/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm) 2024-09-17T03:16:09.788 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-17T03:16:09.797 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-17T03:16:09.817 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-17T03:16:09.817 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:16:09.817 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:16:09.837 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=156 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:16:09.838 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-09-17T03:16:09.838 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:16:09.893 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056010,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=156 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248716k,nr_inodes=812179,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497436k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-17T03:16:09.894 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:16:09.894 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:16:09.913 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=1994716,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=3199368k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20815 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=1599680k,nr_inodes=399920,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=3199368k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:16:09.913 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-09-17T03:16:09.913 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-17T03:16:09.970 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=1994716,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=3199368k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20815 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=1599680k,nr_inodes=399920,mode=700,uid=1000,gid=1333,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=3199368k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-17T03:16:09.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:16:09.970 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-09-17T03:16:09.986 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-17T03:16:09.986 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-17T03:16:09.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:16:09.987 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-17T03:16:10.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:16:10.052 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-17T03:16:10.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:16:10.119 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link delete ceph-brx 2024-09-17T03:16:10.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:16:10.213 DEBUG:teuthology.orchestra.run.smithi149:> ip netns list 2024-09-17T03:16:10.230 INFO:teuthology.orchestra.run.smithi149.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-17T03:16:10.230 INFO:teuthology.orchestra.run.smithi149.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-17T03:16:10.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:16:10.231 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-17T03:16:10.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:16:10.299 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-17T03:16:10.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-17T03:16:10.367 DEBUG:teuthology.orchestra.run.smithi149:> sudo ip link delete ceph-brx 2024-09-17T03:16:10.464 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-17T03:16:10.473 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-17T03:16:10.474 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:16:10.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.593+0000 7f64ff94a640 1 -- 172.21.15.87:0/1561347505 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f64f80cc590 msgr2=0x7f64f815c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:10.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.593+0000 7f64ff94a640 1 --2- 172.21.15.87:0/1561347505 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f64f80cc590 0x7f64f815c470 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f64f4009be0 tx=0x7f64f402a920 comp rx=0 tx=0).stop 2024-09-17T03:16:10.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.593+0000 7f64ff94a640 1 -- 172.21.15.87:0/1561347505 shutdown_connections 2024-09-17T03:16:10.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.593+0000 7f64ff94a640 1 --2- 172.21.15.87:0/1561347505 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f64f80cc590 0x7f64f815c470 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:10.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.593+0000 7f64ff94a640 1 -- 172.21.15.87:0/1561347505 >> 172.21.15.87:0/1561347505 conn(0x7f64f806ee70 msgr2=0x7f64f806f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:10.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.593+0000 7f64ff94a640 1 -- 172.21.15.87:0/1561347505 shutdown_connections 2024-09-17T03:16:10.594 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.593+0000 7f64ff94a640 1 -- 172.21.15.87:0/1561347505 wait complete. 2024-09-17T03:16:10.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.595+0000 7f64ff94a640 1 Processor -- start 2024-09-17T03:16:10.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.595+0000 7f64ff94a640 1 -- start start 2024-09-17T03:16:10.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.597+0000 7f64ff94a640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64f80cc590 0x7f64f80afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:10.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.597+0000 7f64ff94a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64f815f0c0 0x7f64f81ef4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:10.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.597+0000 7f64ff94a640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f64f81f62a0 0x7f64f815dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:10.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.597+0000 7f64ff94a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f64f815f6e0 con 0x7f64f815f0c0 2024-09-17T03:16:10.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.597+0000 7f64ff94a640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f64f80c6b80 con 0x7f64f80cc590 2024-09-17T03:16:10.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.597+0000 7f64ff94a640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f64f80c5610 con 0x7f64f81f62a0 2024-09-17T03:16:10.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.597+0000 7f64fcee5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64f815f0c0 0x7f64f81ef4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:10.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.597+0000 7f64fcee5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64f815f0c0 0x7f64f81ef4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46518/0 (socket says 172.21.15.87:46518) 2024-09-17T03:16:10.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.597+0000 7f64fcee5640 1 -- 172.21.15.87:0/2710720743 learned_addr learned my addr 172.21.15.87:0/2710720743 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:10.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.597+0000 7f64fd6e6640 1 --2- 172.21.15.87:0/2710720743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64f80cc590 0x7f64f80afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:10.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.597+0000 7f64fdee7640 1 --2- 172.21.15.87:0/2710720743 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f64f81f62a0 0x7f64f815dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:10.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.597+0000 7f64fcee5640 1 -- 172.21.15.87:0/2710720743 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f64f81f62a0 msgr2=0x7f64f815dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:10.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.597+0000 7f64fcee5640 1 --2- 172.21.15.87:0/2710720743 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f64f81f62a0 0x7f64f815dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:10.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.597+0000 7f64fcee5640 1 -- 172.21.15.87:0/2710720743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64f80cc590 msgr2=0x7f64f80afc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:10.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.597+0000 7f64fcee5640 1 --2- 172.21.15.87:0/2710720743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64f80cc590 0x7f64f80afc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:10.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.597+0000 7f64fcee5640 1 -- 172.21.15.87:0/2710720743 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f81efa30 con 0x7f64f815f0c0 2024-09-17T03:16:10.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.597+0000 7f64fcee5640 1 --2- 172.21.15.87:0/2710720743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64f815f0c0 0x7f64f81ef4f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f64e800c3a0 tx=0x7f64e800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:10.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.597+0000 7f64edffb640 1 -- 172.21.15.87:0/2710720743 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e80052d0 con 0x7f64f815f0c0 2024-09-17T03:16:10.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.597+0000 7f64edffb640 1 -- 172.21.15.87:0/2710720743 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64e80106a0 con 0x7f64f815f0c0 2024-09-17T03:16:10.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.597+0000 7f64ff94a640 1 -- 172.21.15.87:0/2710720743 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f8142b40 con 0x7f64f815f0c0 2024-09-17T03:16:10.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.597+0000 7f64edffb640 1 -- 172.21.15.87:0/2710720743 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e8010af0 con 0x7f64f815f0c0 2024-09-17T03:16:10.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.598+0000 7f64ff94a640 1 -- 172.21.15.87:0/2710720743 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f8143100 con 0x7f64f815f0c0 2024-09-17T03:16:10.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.599+0000 7f64ff94a640 1 -- 172.21.15.87:0/2710720743 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64f815c470 con 0x7f64f815f0c0 2024-09-17T03:16:10.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.599+0000 7f64edffb640 1 -- 172.21.15.87:0/2710720743 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f64e8010c90 con 0x7f64f815f0c0 2024-09-17T03:16:10.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.599+0000 7f64edffb640 1 --2- 172.21.15.87:0/2710720743 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f64dc078b90 0x7f64dc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:10.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.600+0000 7f64edffb640 1 -- 172.21.15.87:0/2710720743 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f64e809c8f0 con 0x7f64f815f0c0 2024-09-17T03:16:10.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.600+0000 7f64fd6e6640 1 --2- 172.21.15.87:0/2710720743 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f64dc078b90 0x7f64dc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:10.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.600+0000 7f64fd6e6640 1 --2- 172.21.15.87:0/2710720743 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f64dc078b90 0x7f64dc07b0a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f64f4034000 tx=0x7f64f4004ab0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:10.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.604+0000 7f64edffb640 1 -- 172.21.15.87:0/2710720743 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f64e80649e0 con 0x7f64f815f0c0 2024-09-17T03:16:10.777 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.778+0000 7f64ff94a640 1 -- 172.21.15.87:0/2710720743 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f64f81428f0 con 0x7f64dc078b90 2024-09-17T03:16:10.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.779+0000 7f64edffb640 1 -- 172.21.15.87:0/2710720743 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13402 (secure 0 0 0) 0x7f64f81428f0 con 0x7f64dc078b90 2024-09-17T03:16:10.779 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:16:10.779 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-09-17T03:16:10.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.781+0000 7f64ff94a640 1 -- 172.21.15.87:0/2710720743 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f64dc078b90 msgr2=0x7f64dc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:10.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.781+0000 7f64ff94a640 1 --2- 172.21.15.87:0/2710720743 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f64dc078b90 0x7f64dc07b0a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f64f4034000 tx=0x7f64f4004ab0 comp rx=0 tx=0).stop 2024-09-17T03:16:10.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.781+0000 7f64ff94a640 1 -- 172.21.15.87:0/2710720743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64f815f0c0 msgr2=0x7f64f81ef4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:10.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.781+0000 7f64ff94a640 1 --2- 172.21.15.87:0/2710720743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64f815f0c0 0x7f64f81ef4f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f64e800c3a0 tx=0x7f64e800c860 comp rx=0 tx=0).stop 2024-09-17T03:16:10.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.781+0000 7f64ff94a640 1 -- 172.21.15.87:0/2710720743 shutdown_connections 2024-09-17T03:16:10.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.781+0000 7f64ff94a640 1 --2- 172.21.15.87:0/2710720743 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f64dc078b90 0x7f64dc07b0a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:10.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.781+0000 7f64ff94a640 1 --2- 172.21.15.87:0/2710720743 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f64f81f62a0 0x7f64f815dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:10.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.781+0000 7f64ff94a640 1 --2- 172.21.15.87:0/2710720743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64f815f0c0 0x7f64f81ef4f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:10.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.781+0000 7f64ff94a640 1 --2- 172.21.15.87:0/2710720743 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f64f80cc590 0x7f64f80afc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:10.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.781+0000 7f64ff94a640 1 -- 172.21.15.87:0/2710720743 >> 172.21.15.87:0/2710720743 conn(0x7f64f806ee70 msgr2=0x7f64f8072cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:10.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.781+0000 7f64ff94a640 1 -- 172.21.15.87:0/2710720743 shutdown_connections 2024-09-17T03:16:10.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.781+0000 7f64ff94a640 1 -- 172.21.15.87:0/2710720743 wait complete. 2024-09-17T03:16:10.792 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-09-17T03:16:09.446243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230640,"kb_used_data":6100,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744080,"statfs":{"total":773094113280,"available":772857937920,"internally_reserved":0,"allocated":6246400,"data_stored":3275879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14792,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-65,"num_write_kb":-36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001599"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":166,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T03:16:06.515747+0000","last_change":"2024-09-17T03:14:04.862167+0000","last_active":"2024-09-17T03:16:06.515747+0000","last_peered":"2024-09-17T03:16:06.515747+0000","last_clean":"2024-09-17T03:16:06.515747+0000","last_became_active":"2024-09-17T03:14:04.861731+0000","last_became_peered":"2024-09-17T03:14:04.861731+0000","last_unstale":"2024-09-17T03:16:06.515747+0000","last_undegraded":"2024-09-17T03:16:06.515747+0000","last_fullsized":"2024-09-17T03:16:06.515747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_clean_scrub_stamp":"2024-09-17T03:14:03.847721+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:32.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509470,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":258709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":259235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1245184,"data_stored":865409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509470,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":258743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":258687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1220608,"data_stored":849091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":259235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":643072,"data_stored":266770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:16:10.793 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:16:10.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.906+0000 7f740197b640 1 -- 172.21.15.87:0/779090250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73fc0d4150 msgr2=0x7f73fc0d4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:10.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.906+0000 7f740197b640 1 --2- 172.21.15.87:0/779090250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73fc0d4150 0x7f73fc0d4560 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f73fc08c800 tx=0x7f73f002a4d0 comp rx=0 tx=0).stop 2024-09-17T03:16:10.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.906+0000 7f740197b640 1 -- 172.21.15.87:0/779090250 shutdown_connections 2024-09-17T03:16:10.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.906+0000 7f740197b640 1 --2- 172.21.15.87:0/779090250 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73fc0d4150 0x7f73fc0d4560 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:10.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.906+0000 7f740197b640 1 --2- 172.21.15.87:0/779090250 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f73fc0d0e60 0x7f73fc0d1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:10.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.906+0000 7f740197b640 1 --2- 172.21.15.87:0/779090250 >> v2:172.21.15.149:3300/0 conn(0x7f73fc0d2c50 0x7f73fc0d3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:10.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.906+0000 7f740197b640 1 -- 172.21.15.87:0/779090250 >> 172.21.15.87:0/779090250 conn(0x7f73fc08dae0 msgr2=0x7f73fc09d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:10.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.906+0000 7f740197b640 1 -- 172.21.15.87:0/779090250 shutdown_connections 2024-09-17T03:16:10.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.906+0000 7f740197b640 1 -- 172.21.15.87:0/779090250 wait complete. 2024-09-17T03:16:10.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.907+0000 7f740197b640 1 Processor -- start 2024-09-17T03:16:10.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.908+0000 7f740197b640 1 -- start start 2024-09-17T03:16:10.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.909+0000 7f740197b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73fc0d0e60 0x7f73fc0c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:10.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.909+0000 7f740197b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73fc0d2c50 0x7f73fc0cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:10.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.909+0000 7f740197b640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f73fc0d4150 0x7f73fc1f8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:10.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.909+0000 7f740197b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f73fc0d17b0 con 0x7f73fc0d2c50 2024-09-17T03:16:10.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.909+0000 7f740197b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f73fc1622b0 con 0x7f73fc0d0e60 2024-09-17T03:16:10.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.909+0000 7f740197b640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f73fc0d1930 con 0x7f73fc0d4150 2024-09-17T03:16:10.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.909+0000 7f73fa7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73fc0d2c50 0x7f73fc0cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:10.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.909+0000 7f73fa7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73fc0d2c50 0x7f73fc0cdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46538/0 (socket says 172.21.15.87:46538) 2024-09-17T03:16:10.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.909+0000 7f73fa7fc640 1 -- 172.21.15.87:0/1271809345 learned_addr learned my addr 172.21.15.87:0/1271809345 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:10.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.909+0000 7f73faffd640 1 --2- 172.21.15.87:0/1271809345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73fc0d0e60 0x7f73fc0c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:10.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.909+0000 7f73fb7fe640 1 --2- 172.21.15.87:0/1271809345 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f73fc0d4150 0x7f73fc1f8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:10.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.909+0000 7f73fa7fc640 1 -- 172.21.15.87:0/1271809345 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f73fc0d4150 msgr2=0x7f73fc1f8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:10.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.909+0000 7f73fa7fc640 1 --2- 172.21.15.87:0/1271809345 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f73fc0d4150 0x7f73fc1f8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:10.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.909+0000 7f73fa7fc640 1 -- 172.21.15.87:0/1271809345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73fc0d0e60 msgr2=0x7f73fc0c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:10.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.909+0000 7f73fa7fc640 1 --2- 172.21.15.87:0/1271809345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73fc0d0e60 0x7f73fc0c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:10.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.909+0000 7f73fa7fc640 1 -- 172.21.15.87:0/1271809345 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73fc0ca730 con 0x7f73fc0d2c50 2024-09-17T03:16:10.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.910+0000 7f73fa7fc640 1 --2- 172.21.15.87:0/1271809345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73fc0d2c50 0x7f73fc0cdeb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f73f00099c0 tx=0x7f73f00039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:10.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.910+0000 7f73e3fff640 1 -- 172.21.15.87:0/1271809345 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f003e030 con 0x7f73fc0d2c50 2024-09-17T03:16:10.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.910+0000 7f73e3fff640 1 -- 172.21.15.87:0/1271809345 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73f00047a0 con 0x7f73fc0d2c50 2024-09-17T03:16:10.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.910+0000 7f740197b640 1 -- 172.21.15.87:0/1271809345 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73fc0caa10 con 0x7f73fc0d2c50 2024-09-17T03:16:10.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.910+0000 7f73e3fff640 1 -- 172.21.15.87:0/1271809345 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f0035040 con 0x7f73fc0d2c50 2024-09-17T03:16:10.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.910+0000 7f740197b640 1 -- 172.21.15.87:0/1271809345 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73fc0b2050 con 0x7f73fc0d2c50 2024-09-17T03:16:10.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.911+0000 7f740197b640 1 -- 172.21.15.87:0/1271809345 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73fc0d3060 con 0x7f73fc0d2c50 2024-09-17T03:16:10.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.911+0000 7f73e3fff640 1 -- 172.21.15.87:0/1271809345 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f73f0041030 con 0x7f73fc0d2c50 2024-09-17T03:16:10.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.912+0000 7f73e3fff640 1 --2- 172.21.15.87:0/1271809345 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f73e8078b90 0x7f73e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:10.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.912+0000 7f73e3fff640 1 -- 172.21.15.87:0/1271809345 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f73f00bb0e0 con 0x7f73fc0d2c50 2024-09-17T03:16:10.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.912+0000 7f73faffd640 1 --2- 172.21.15.87:0/1271809345 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f73e8078b90 0x7f73e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:10.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.913+0000 7f73faffd640 1 --2- 172.21.15.87:0/1271809345 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f73e8078b90 0x7f73e807b0a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f73ec005fd0 tx=0x7f73ec0095e0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:10.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:10.917+0000 7f73e3fff640 1 -- 172.21.15.87:0/1271809345 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f73f0083260 con 0x7f73fc0d2c50 2024-09-17T03:16:11.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.088+0000 7f740197b640 1 -- 172.21.15.87:0/1271809345 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f73fc0ca4c0 con 0x7f73e8078b90 2024-09-17T03:16:11.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.089+0000 7f73e3fff640 1 -- 172.21.15.87:0/1271809345 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13402 (secure 0 0 0) 0x7f73fc0ca4c0 con 0x7f73e8078b90 2024-09-17T03:16:11.089 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:16:11.089 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-09-17T03:16:11.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.091+0000 7f740197b640 1 -- 172.21.15.87:0/1271809345 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f73e8078b90 msgr2=0x7f73e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:11.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.091+0000 7f740197b640 1 --2- 172.21.15.87:0/1271809345 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f73e8078b90 0x7f73e807b0a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f73ec005fd0 tx=0x7f73ec0095e0 comp rx=0 tx=0).stop 2024-09-17T03:16:11.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.091+0000 7f740197b640 1 -- 172.21.15.87:0/1271809345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73fc0d2c50 msgr2=0x7f73fc0cdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:11.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.091+0000 7f740197b640 1 --2- 172.21.15.87:0/1271809345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73fc0d2c50 0x7f73fc0cdeb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f73f00099c0 tx=0x7f73f00039e0 comp rx=0 tx=0).stop 2024-09-17T03:16:11.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.091+0000 7f740197b640 1 -- 172.21.15.87:0/1271809345 shutdown_connections 2024-09-17T03:16:11.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.091+0000 7f740197b640 1 --2- 172.21.15.87:0/1271809345 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f73e8078b90 0x7f73e807b0a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:11.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.091+0000 7f740197b640 1 --2- 172.21.15.87:0/1271809345 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f73fc0d4150 0x7f73fc1f8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:11.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.091+0000 7f740197b640 1 --2- 172.21.15.87:0/1271809345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f73fc0d2c50 0x7f73fc0cdeb0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:11.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.091+0000 7f740197b640 1 --2- 172.21.15.87:0/1271809345 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f73fc0d0e60 0x7f73fc0c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:11.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.091+0000 7f740197b640 1 -- 172.21.15.87:0/1271809345 >> 172.21.15.87:0/1271809345 conn(0x7f73fc08dae0 msgr2=0x7f73fc15c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:11.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.091+0000 7f740197b640 1 -- 172.21.15.87:0/1271809345 shutdown_connections 2024-09-17T03:16:11.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.091+0000 7f740197b640 1 -- 172.21.15.87:0/1271809345 wait complete. 2024-09-17T03:16:11.102 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-09-17T03:16:09.446243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230640,"kb_used_data":6100,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744080,"statfs":{"total":773094113280,"available":772857937920,"internally_reserved":0,"allocated":6246400,"data_stored":3275879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14792,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-65,"num_write_kb":-36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001599"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":166,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T03:16:06.515747+0000","last_change":"2024-09-17T03:14:04.862167+0000","last_active":"2024-09-17T03:16:06.515747+0000","last_peered":"2024-09-17T03:16:06.515747+0000","last_clean":"2024-09-17T03:16:06.515747+0000","last_became_active":"2024-09-17T03:14:04.861731+0000","last_became_peered":"2024-09-17T03:14:04.861731+0000","last_unstale":"2024-09-17T03:16:06.515747+0000","last_undegraded":"2024-09-17T03:16:06.515747+0000","last_fullsized":"2024-09-17T03:16:06.515747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_clean_scrub_stamp":"2024-09-17T03:14:03.847721+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:32.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509470,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":258709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":259235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1245184,"data_stored":865409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509470,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":258743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":258687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1220608,"data_stored":849091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":259235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":643072,"data_stored":266770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:16:11.103 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-17T03:16:11.103 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:16:11.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.215+0000 7fbcc6ab4640 1 -- 172.21.15.87:0/3408706120 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbcc00b76d0 msgr2=0x7fbcc00cee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:11.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.215+0000 7fbcc6ab4640 1 --2- 172.21.15.87:0/3408706120 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbcc00b76d0 0x7fbcc00cee00 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00b38e0 tx=0x7fbcbc02a530 comp rx=0 tx=0).stop 2024-09-17T03:16:11.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.215+0000 7fbcc6ab4640 1 -- 172.21.15.87:0/3408706120 shutdown_connections 2024-09-17T03:16:11.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.215+0000 7fbcc6ab4640 1 --2- 172.21.15.87:0/3408706120 >> v2:172.21.15.149:3300/0 conn(0x7fbcc00ccf80 0x7fbcc00cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:11.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.215+0000 7fbcc6ab4640 1 --2- 172.21.15.87:0/3408706120 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbcc00b76d0 0x7fbcc00cee00 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:11.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.215+0000 7fbcc6ab4640 1 -- 172.21.15.87:0/3408706120 >> 172.21.15.87:0/3408706120 conn(0x7fbcc00d2230 msgr2=0x7fbcc00d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:11.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.215+0000 7fbcc6ab4640 1 -- 172.21.15.87:0/3408706120 shutdown_connections 2024-09-17T03:16:11.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.215+0000 7fbcc6ab4640 1 -- 172.21.15.87:0/3408706120 wait complete. 2024-09-17T03:16:11.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.217+0000 7fbcc6ab4640 1 Processor -- start 2024-09-17T03:16:11.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.217+0000 7fbcc6ab4640 1 -- start start 2024-09-17T03:16:11.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.219+0000 7fbcc6ab4640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbcc00b76d0 0x7fbcc00b5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:11.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.219+0000 7fbcc6ab4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcc00ccf80 0x7fbcc009f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:11.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.219+0000 7fbcc6ab4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbcc00cf340 0x7fbcc01dbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:11.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.219+0000 7fbcc6ab4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc00d13d0 con 0x7fbcc00cf340 2024-09-17T03:16:11.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.219+0000 7fbcc6ab4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc00d1250 con 0x7fbcc00ccf80 2024-09-17T03:16:11.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.219+0000 7fbcc6ab4640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fbcc00b19d0 con 0x7fbcc00b76d0 2024-09-17T03:16:11.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.219+0000 7fbcc5051640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbcc00cf340 0x7fbcc01dbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:11.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.219+0000 7fbcc5051640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbcc00cf340 0x7fbcc01dbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46548/0 (socket says 172.21.15.87:46548) 2024-09-17T03:16:11.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.219+0000 7fbcc5051640 1 -- 172.21.15.87:0/2106888571 learned_addr learned my addr 172.21.15.87:0/2106888571 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:11.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.219+0000 7fbcc4850640 1 --2- 172.21.15.87:0/2106888571 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbcc00b76d0 0x7fbcc00b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:11.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.219+0000 7fbcb7fff640 1 --2- 172.21.15.87:0/2106888571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcc00ccf80 0x7fbcc009f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:11.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.219+0000 7fbcc5051640 1 -- 172.21.15.87:0/2106888571 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbcc00b76d0 msgr2=0x7fbcc00b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:11.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.219+0000 7fbcc5051640 1 --2- 172.21.15.87:0/2106888571 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbcc00b76d0 0x7fbcc00b5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:11.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.219+0000 7fbcc5051640 1 -- 172.21.15.87:0/2106888571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcc00ccf80 msgr2=0x7fbcc009f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:11.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.219+0000 7fbcc5051640 1 --2- 172.21.15.87:0/2106888571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcc00ccf80 0x7fbcc009f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:11.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.219+0000 7fbcc5051640 1 -- 172.21.15.87:0/2106888571 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc01f33c0 con 0x7fbcc00cf340 2024-09-17T03:16:11.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.219+0000 7fbcb7fff640 1 --2- 172.21.15.87:0/2106888571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcc00ccf80 0x7fbcc009f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:16:11.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.219+0000 7fbcc5051640 1 --2- 172.21.15.87:0/2106888571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbcc00cf340 0x7fbcc01dbb40 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fbcb800e9c0 tx=0x7fbcb800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:11.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.219+0000 7fbcb57fa640 1 -- 172.21.15.87:0/2106888571 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8004d20 con 0x7fbcc00cf340 2024-09-17T03:16:11.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.219+0000 7fbcb57fa640 1 -- 172.21.15.87:0/2106888571 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcb800cb20 con 0x7fbcc00cf340 2024-09-17T03:16:11.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.219+0000 7fbcc6ab4640 1 -- 172.21.15.87:0/2106888571 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc00af500 con 0x7fbcc00cf340 2024-09-17T03:16:11.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.219+0000 7fbcb57fa640 1 -- 172.21.15.87:0/2106888571 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8005020 con 0x7fbcc00cf340 2024-09-17T03:16:11.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.220+0000 7fbcc6ab4640 1 -- 172.21.15.87:0/2106888571 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc00afaa0 con 0x7fbcc00cf340 2024-09-17T03:16:11.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.221+0000 7fbcc6ab4640 1 -- 172.21.15.87:0/2106888571 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcc00cee00 con 0x7fbcc00cf340 2024-09-17T03:16:11.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.221+0000 7fbcb57fa640 1 -- 172.21.15.87:0/2106888571 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbcb80051c0 con 0x7fbcc00cf340 2024-09-17T03:16:11.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.222+0000 7fbcb57fa640 1 --2- 172.21.15.87:0/2106888571 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbcac078b90 0x7fbcac07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:11.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.222+0000 7fbcb57fa640 1 -- 172.21.15.87:0/2106888571 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7fbcb809b300 con 0x7fbcc00cf340 2024-09-17T03:16:11.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.222+0000 7fbcc4850640 1 --2- 172.21.15.87:0/2106888571 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbcac078b90 0x7fbcac07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:11.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.223+0000 7fbcc4850640 1 --2- 172.21.15.87:0/2106888571 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbcac078b90 0x7fbcac07b0a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc034000 tx=0x7fbcbc004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:11.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.227+0000 7fbcb57fa640 1 -- 172.21.15.87:0/2106888571 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fbcc01f33c0 con 0x7fbcc00cf340 2024-09-17T03:16:11.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.400+0000 7fbcc6ab4640 1 -- 172.21.15.87:0/2106888571 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbcc00cda40 con 0x7fbcac078b90 2024-09-17T03:16:11.401 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.401+0000 7fbcb57fa640 1 -- 172.21.15.87:0/2106888571 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13402 (secure 0 0 0) 0x7fbcc00cda40 con 0x7fbcac078b90 2024-09-17T03:16:11.401 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:16:11.402 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-09-17T03:16:11.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.404+0000 7fbcc6ab4640 1 -- 172.21.15.87:0/2106888571 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbcac078b90 msgr2=0x7fbcac07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:11.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.404+0000 7fbcc6ab4640 1 --2- 172.21.15.87:0/2106888571 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbcac078b90 0x7fbcac07b0a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc034000 tx=0x7fbcbc004730 comp rx=0 tx=0).stop 2024-09-17T03:16:11.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.404+0000 7fbcc6ab4640 1 -- 172.21.15.87:0/2106888571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbcc00cf340 msgr2=0x7fbcc01dbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:11.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.404+0000 7fbcc6ab4640 1 --2- 172.21.15.87:0/2106888571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbcc00cf340 0x7fbcc01dbb40 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fbcb800e9c0 tx=0x7fbcb800ee80 comp rx=0 tx=0).stop 2024-09-17T03:16:11.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.404+0000 7fbcc6ab4640 1 -- 172.21.15.87:0/2106888571 shutdown_connections 2024-09-17T03:16:11.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.404+0000 7fbcc6ab4640 1 --2- 172.21.15.87:0/2106888571 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fbcac078b90 0x7fbcac07b0a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:11.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.404+0000 7fbcc6ab4640 1 --2- 172.21.15.87:0/2106888571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbcc00cf340 0x7fbcc01dbb40 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:11.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.404+0000 7fbcc6ab4640 1 --2- 172.21.15.87:0/2106888571 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fbcc00ccf80 0x7fbcc009f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:11.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.404+0000 7fbcc6ab4640 1 --2- 172.21.15.87:0/2106888571 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fbcc00b76d0 0x7fbcc00b5290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:11.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.404+0000 7fbcc6ab4640 1 -- 172.21.15.87:0/2106888571 >> 172.21.15.87:0/2106888571 conn(0x7fbcc00d2230 msgr2=0x7fbcc00d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:11.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.404+0000 7fbcc6ab4640 1 -- 172.21.15.87:0/2106888571 shutdown_connections 2024-09-17T03:16:11.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.404+0000 7fbcc6ab4640 1 -- 172.21.15.87:0/2106888571 wait complete. 2024-09-17T03:16:11.415 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-09-17T03:16:09.446243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230640,"kb_used_data":6100,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744080,"statfs":{"total":773094113280,"available":772857937920,"internally_reserved":0,"allocated":6246400,"data_stored":3275879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14792,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-65,"num_write_kb":-36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001599"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":166,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-17T03:16:06.515747+0000","last_change":"2024-09-17T03:14:04.862167+0000","last_active":"2024-09-17T03:16:06.515747+0000","last_peered":"2024-09-17T03:16:06.515747+0000","last_clean":"2024-09-17T03:16:06.515747+0000","last_became_active":"2024-09-17T03:14:04.861731+0000","last_became_peered":"2024-09-17T03:14:04.861731+0000","last_unstale":"2024-09-17T03:16:06.515747+0000","last_undegraded":"2024-09-17T03:16:06.515747+0000","last_fullsized":"2024-09-17T03:16:06.515747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_clean_scrub_stamp":"2024-09-17T03:14:03.847721+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:32.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509470,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":258709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509470,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":259235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1245184,"data_stored":865409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509470,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":258743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509470,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":258687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509470,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28220,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343620,"statfs":{"total":96636764160,"available":96607866880,"internally_reserved":0,"allocated":1220608,"data_stored":849091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":626688,"data_stored":259235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":643072,"data_stored":266770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:16:11.415 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-17T03:16:11.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.528+0000 7f6ccf1ad640 1 -- 172.21.15.87:0/1285775710 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6cc80b3d60 msgr2=0x7f6cc80b4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:11.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.528+0000 7f6ccf1ad640 1 --2- 172.21.15.87:0/1285775710 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6cc80b3d60 0x7f6cc80b4170 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8005fd0 tx=0x7f6cb802a540 comp rx=0 tx=0).stop 2024-09-17T03:16:11.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.528+0000 7f6ccf1ad640 1 -- 172.21.15.87:0/1285775710 shutdown_connections 2024-09-17T03:16:11.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.528+0000 7f6ccf1ad640 1 --2- 172.21.15.87:0/1285775710 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6cc80b3d60 0x7f6cc80b4170 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:11.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.528+0000 7f6ccf1ad640 1 --2- 172.21.15.87:0/1285775710 >> v2:172.21.15.149:3300/0 conn(0x7f6cc80b7f10 0x7f6cc815c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:11.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.528+0000 7f6ccf1ad640 1 -- 172.21.15.87:0/1285775710 >> 172.21.15.87:0/1285775710 conn(0x7f6cc8069330 msgr2=0x7f6cc80ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:11.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.528+0000 7f6ccf1ad640 1 -- 172.21.15.87:0/1285775710 shutdown_connections 2024-09-17T03:16:11.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.528+0000 7f6ccf1ad640 1 -- 172.21.15.87:0/1285775710 wait complete. 2024-09-17T03:16:11.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.530+0000 7f6ccf1ad640 1 Processor -- start 2024-09-17T03:16:11.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.530+0000 7f6ccf1ad640 1 -- start start 2024-09-17T03:16:11.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.531+0000 7f6ccf1ad640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6cc80710e0 0x7f6cc80c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:11.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.531+0000 7f6ccf1ad640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6cc80b3d60 0x7f6cc815f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:11.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.531+0000 7f6ccf1ad640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cc80b7f10 0x7f6cc815fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:11.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.531+0000 7f6ccf1ad640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc80b7b10 con 0x7f6cc80b7f10 2024-09-17T03:16:11.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.531+0000 7f6ccf1ad640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc80d1c70 con 0x7f6cc80b3d60 2024-09-17T03:16:11.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.531+0000 7f6ccf1ad640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6cc80b7c90 con 0x7f6cc80710e0 2024-09-17T03:16:11.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.531+0000 7f6ccd74a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cc80b7f10 0x7f6cc815fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:11.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.531+0000 7f6ccd74a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cc80b7f10 0x7f6cc815fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46554/0 (socket says 172.21.15.87:46554) 2024-09-17T03:16:11.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.531+0000 7f6ccd74a640 1 -- 172.21.15.87:0/2295755336 learned_addr learned my addr 172.21.15.87:0/2295755336 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:11.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.532+0000 7f6cbffff640 1 --2- 172.21.15.87:0/2295755336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6cc80b3d60 0x7f6cc815f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:11.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.532+0000 7f6cccf49640 1 --2- 172.21.15.87:0/2295755336 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6cc80710e0 0x7f6cc80c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:11.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.532+0000 7f6ccd74a640 1 -- 172.21.15.87:0/2295755336 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6cc80710e0 msgr2=0x7f6cc80c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:11.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.532+0000 7f6ccd74a640 1 --2- 172.21.15.87:0/2295755336 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6cc80710e0 0x7f6cc80c9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:11.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.532+0000 7f6ccd74a640 1 -- 172.21.15.87:0/2295755336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6cc80b3d60 msgr2=0x7f6cc815f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:11.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.532+0000 7f6ccd74a640 1 --2- 172.21.15.87:0/2295755336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6cc80b3d60 0x7f6cc815f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:11.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.532+0000 7f6ccd74a640 1 -- 172.21.15.87:0/2295755336 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc814a440 con 0x7f6cc80b7f10 2024-09-17T03:16:11.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.532+0000 7f6ccd74a640 1 --2- 172.21.15.87:0/2295755336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cc80b7f10 0x7f6cc815fc20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000c700 tx=0x7f6cc000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:11.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.532+0000 7f6cbd7fa640 1 -- 172.21.15.87:0/2295755336 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0003b70 con 0x7f6cc80b7f10 2024-09-17T03:16:11.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.532+0000 7f6cbd7fa640 1 -- 172.21.15.87:0/2295755336 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cc0003d10 con 0x7f6cc80b7f10 2024-09-17T03:16:11.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.532+0000 7f6cbd7fa640 1 -- 172.21.15.87:0/2295755336 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc000aab0 con 0x7f6cc80b7f10 2024-09-17T03:16:11.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.532+0000 7f6ccf1ad640 1 -- 172.21.15.87:0/2295755336 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cc81498c0 con 0x7f6cc80b7f10 2024-09-17T03:16:11.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.532+0000 7f6ccf1ad640 1 -- 172.21.15.87:0/2295755336 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cc8149e80 con 0x7f6cc80b7f10 2024-09-17T03:16:11.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.533+0000 7f6ccf1ad640 1 -- 172.21.15.87:0/2295755336 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cc8071e30 con 0x7f6cc80b7f10 2024-09-17T03:16:11.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.534+0000 7f6cbd7fa640 1 -- 172.21.15.87:0/2295755336 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6cc000ac50 con 0x7f6cc80b7f10 2024-09-17T03:16:11.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.534+0000 7f6cbd7fa640 1 --2- 172.21.15.87:0/2295755336 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6cb4078b90 0x7f6cb407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:11.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.535+0000 7f6cbd7fa640 1 -- 172.21.15.87:0/2295755336 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f6cc009a6f0 con 0x7f6cc80b7f10 2024-09-17T03:16:11.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.535+0000 7f6cccf49640 1 --2- 172.21.15.87:0/2295755336 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6cb4078b90 0x7f6cb407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:11.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.535+0000 7f6cccf49640 1 --2- 172.21.15.87:0/2295755336 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6cb4078b90 0x7f6cb407b0a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009b30 tx=0x7f6cc4005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:11.538 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.539+0000 7f6cbd7fa640 1 -- 172.21.15.87:0/2295755336 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f6cc80b46b0 con 0x7f6cc80b7f10 2024-09-17T03:16:11.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.716+0000 7f6ccf1ad640 1 -- 172.21.15.87:0/2295755336 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6cc8149650 con 0x7f6cc80b7f10 2024-09-17T03:16:11.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.717+0000 7f6cbd7fa640 1 -- 172.21.15.87:0/2295755336 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+13058 (secure 0 0 0) 0x7f6cc00627e0 con 0x7f6cc80b7f10 2024-09-17T03:16:11.717 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:16:11.717 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":55,"fsid":"7b2553aa-e562-4da8-b175-8e91f460da55","created":"2024-09-17T03:13:48.953421+0000","modified":"2024-09-17T03:16:08.580820+0000","last_up_change":"2024-09-17T03:14:01.833605+0000","last_in_change":"2024-09-17T03:13:52.569440+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-17T03:14:03.433404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22b83449-856f-4b82-b290-cd0e9e438bbf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6825","nonce":2430409749}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6827","nonce":2430409749}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6831","nonce":2430409749}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":2430409749},{"type":"v1","addr":"172.21.15.87:6829","nonce":2430409749}]},"public_addr":"172.21.15.87:6825/2430409749","cluster_addr":"172.21.15.87:6827/2430409749","heartbeat_back_addr":"172.21.15.87:6831/2430409749","heartbeat_front_addr":"172.21.15.87:6829/2430409749","state":["exists","up"]},{"osd":1,"uuid":"5268ac87-a2b3-4211-bedd-2a869a93511d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6809","nonce":3467605435}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6811","nonce":3467605435}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6815","nonce":3467605435}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":3467605435},{"type":"v1","addr":"172.21.15.87:6813","nonce":3467605435}]},"public_addr":"172.21.15.87:6809/3467605435","cluster_addr":"172.21.15.87:6811/3467605435","heartbeat_back_addr":"172.21.15.87:6815/3467605435","heartbeat_front_addr":"172.21.15.87:6813/3467605435","state":["exists","up"]},{"osd":2,"uuid":"a5923363-d620-4f42-8764-b29e6c3095cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6801","nonce":1450834766}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6803","nonce":1450834766}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6807","nonce":1450834766}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":1450834766},{"type":"v1","addr":"172.21.15.87:6805","nonce":1450834766}]},"public_addr":"172.21.15.87:6801/1450834766","cluster_addr":"172.21.15.87:6803/1450834766","heartbeat_back_addr":"172.21.15.87:6807/1450834766","heartbeat_front_addr":"172.21.15.87:6805/1450834766","state":["exists","up"]},{"osd":3,"uuid":"3ebd41e0-5f1a-4b97-821d-0ef0ff1b7874","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6817","nonce":4253161216}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6819","nonce":4253161216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6823","nonce":4253161216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":4253161216},{"type":"v1","addr":"172.21.15.87:6821","nonce":4253161216}]},"public_addr":"172.21.15.87:6817/4253161216","cluster_addr":"172.21.15.87:6819/4253161216","heartbeat_back_addr":"172.21.15.87:6823/4253161216","heartbeat_front_addr":"172.21.15.87:6821/4253161216","state":["exists","up"]},{"osd":4,"uuid":"080b1041-3fbe-462c-b735-24f23d52255e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6802","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6803","nonce":2414907089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6805","nonce":2414907089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6809","nonce":2414907089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6806","nonce":2414907089},{"type":"v1","addr":"172.21.15.149:6807","nonce":2414907089}]},"public_addr":"172.21.15.149:6803/2414907089","cluster_addr":"172.21.15.149:6805/2414907089","heartbeat_back_addr":"172.21.15.149:6809/2414907089","heartbeat_front_addr":"172.21.15.149:6807/2414907089","state":["exists","up"]},{"osd":5,"uuid":"d85dddc7-492e-45f6-a160-ffa21b6e5e2d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6826","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6827","nonce":2182999530}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6829","nonce":2182999530}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6833","nonce":2182999530}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6830","nonce":2182999530},{"type":"v1","addr":"172.21.15.149:6831","nonce":2182999530}]},"public_addr":"172.21.15.149:6827/2182999530","cluster_addr":"172.21.15.149:6829/2182999530","heartbeat_back_addr":"172.21.15.149:6833/2182999530","heartbeat_front_addr":"172.21.15.149:6831/2182999530","state":["exists","up"]},{"osd":6,"uuid":"a8a34250-28ce-47f3-a1a4-e298374e4688","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6810","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6811","nonce":4289519146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6813","nonce":4289519146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6817","nonce":4289519146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6814","nonce":4289519146},{"type":"v1","addr":"172.21.15.149:6815","nonce":4289519146}]},"public_addr":"172.21.15.149:6811/4289519146","cluster_addr":"172.21.15.149:6813/4289519146","heartbeat_back_addr":"172.21.15.149:6817/4289519146","heartbeat_front_addr":"172.21.15.149:6815/4289519146","state":["exists","up"]},{"osd":7,"uuid":"33815d16-b7b5-4cb4-80e1-de8012498ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6818","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6819","nonce":2771010477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6821","nonce":2771010477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6825","nonce":2771010477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6822","nonce":2771010477},{"type":"v1","addr":"172.21.15.149:6823","nonce":2771010477}]},"public_addr":"172.21.15.149:6819/2771010477","cluster_addr":"172.21.15.149:6821/2771010477","heartbeat_back_addr":"172.21.15.149:6825/2771010477","heartbeat_front_addr":"172.21.15.149:6823/2771010477","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.313297+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.642861+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.654781+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.302606+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:58.385791+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.915266+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.121365+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-17T03:13:59.394432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.149:6837/3562056427":"2024-09-18T03:16:05.507230+0000","172.21.15.149:6835/3562056427":"2024-09-18T03:16:05.507230+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-17T03:16:11.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.719+0000 7f6ccf1ad640 1 -- 172.21.15.87:0/2295755336 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6cb4078b90 msgr2=0x7f6cb407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:11.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.719+0000 7f6ccf1ad640 1 --2- 172.21.15.87:0/2295755336 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6cb4078b90 0x7f6cb407b0a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009b30 tx=0x7f6cc4005f90 comp rx=0 tx=0).stop 2024-09-17T03:16:11.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.719+0000 7f6ccf1ad640 1 -- 172.21.15.87:0/2295755336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cc80b7f10 msgr2=0x7f6cc815fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:11.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.719+0000 7f6ccf1ad640 1 --2- 172.21.15.87:0/2295755336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cc80b7f10 0x7f6cc815fc20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6cc000c700 tx=0x7f6cc000cbc0 comp rx=0 tx=0).stop 2024-09-17T03:16:11.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.719+0000 7f6ccf1ad640 1 -- 172.21.15.87:0/2295755336 shutdown_connections 2024-09-17T03:16:11.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.719+0000 7f6ccf1ad640 1 --2- 172.21.15.87:0/2295755336 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6cb4078b90 0x7f6cb407b0a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:11.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.719+0000 7f6ccf1ad640 1 --2- 172.21.15.87:0/2295755336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cc80b7f10 0x7f6cc815fc20 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:11.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.719+0000 7f6ccf1ad640 1 --2- 172.21.15.87:0/2295755336 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6cc80b3d60 0x7f6cc815f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:11.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.719+0000 7f6ccf1ad640 1 --2- 172.21.15.87:0/2295755336 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6cc80710e0 0x7f6cc80c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:11.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.719+0000 7f6ccf1ad640 1 -- 172.21.15.87:0/2295755336 >> 172.21.15.87:0/2295755336 conn(0x7f6cc8069330 msgr2=0x7f6cc80b6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:11.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.720+0000 7f6ccf1ad640 1 -- 172.21.15.87:0/2295755336 shutdown_connections 2024-09-17T03:16:11.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:11.720+0000 7f6ccf1ad640 1 -- 172.21.15.87:0/2295755336 wait complete. 2024-09-17T03:16:12.732 INFO:tasks.ceph:Scrubbing osd.0 2024-09-17T03:16:12.732 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-17T03:16:12.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.863+0000 7f6e47884640 1 -- 172.21.15.87:0/1395134696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e400b9ae0 msgr2=0x7f6e40160390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:12.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.863+0000 7f6e47884640 1 --2- 172.21.15.87:0/1395134696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e400b9ae0 0x7f6e40160390 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009820 tx=0x7f6e3c02a480 comp rx=0 tx=0).stop 2024-09-17T03:16:12.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.863+0000 7f6e47884640 1 -- 172.21.15.87:0/1395134696 shutdown_connections 2024-09-17T03:16:12.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.863+0000 7f6e47884640 1 --2- 172.21.15.87:0/1395134696 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6e400cfaa0 0x7f6e40162590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:12.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.863+0000 7f6e47884640 1 --2- 172.21.15.87:0/1395134696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e400b9ae0 0x7f6e40160390 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:12.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.863+0000 7f6e47884640 1 -- 172.21.15.87:0/1395134696 >> 172.21.15.87:0/1395134696 conn(0x7f6e400d4510 msgr2=0x7f6e400d4920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:12.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.863+0000 7f6e47884640 1 -- 172.21.15.87:0/1395134696 shutdown_connections 2024-09-17T03:16:12.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.863+0000 7f6e47884640 1 -- 172.21.15.87:0/1395134696 wait complete. 2024-09-17T03:16:12.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.865+0000 7f6e47884640 1 Processor -- start 2024-09-17T03:16:12.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.865+0000 7f6e47884640 1 -- start start 2024-09-17T03:16:12.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.866+0000 7f6e47884640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6e400b33c0 0x7f6e400c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:12.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.866+0000 7f6e47884640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e400b9ae0 0x7f6e401f9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:12.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.866+0000 7f6e47884640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e400cfaa0 0x7f6e400b66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:12.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.866+0000 7f6e47884640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6e40162e60 con 0x7f6e400cfaa0 2024-09-17T03:16:12.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.866+0000 7f6e47884640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6e40162ce0 con 0x7f6e400b9ae0 2024-09-17T03:16:12.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.866+0000 7f6e47884640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6e40162fe0 con 0x7f6e400b33c0 2024-09-17T03:16:12.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.866+0000 7f6e45e21640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e400cfaa0 0x7f6e400b66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:12.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.866+0000 7f6e45e21640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e400cfaa0 0x7f6e400b66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46568/0 (socket says 172.21.15.87:46568) 2024-09-17T03:16:12.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.866+0000 7f6e45e21640 1 -- 172.21.15.87:0/2024900568 learned_addr learned my addr 172.21.15.87:0/2024900568 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:12.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.866+0000 7f6e45620640 1 --2- 172.21.15.87:0/2024900568 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6e400b33c0 0x7f6e400c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:12.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.866+0000 7f6e44e1f640 1 --2- 172.21.15.87:0/2024900568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e400b9ae0 0x7f6e401f9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:12.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.867+0000 7f6e45e21640 1 -- 172.21.15.87:0/2024900568 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6e400b33c0 msgr2=0x7f6e400c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:12.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.867+0000 7f6e45e21640 1 --2- 172.21.15.87:0/2024900568 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6e400b33c0 0x7f6e400c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:12.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.867+0000 7f6e45e21640 1 -- 172.21.15.87:0/2024900568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e400b9ae0 msgr2=0x7f6e401f9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:12.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.867+0000 7f6e45e21640 1 --2- 172.21.15.87:0/2024900568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e400b9ae0 0x7f6e401f9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:12.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.867+0000 7f6e45e21640 1 -- 172.21.15.87:0/2024900568 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e401f5e90 con 0x7f6e400cfaa0 2024-09-17T03:16:12.869 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.867+0000 7f6e44e1f640 1 --2- 172.21.15.87:0/2024900568 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6e400b9ae0 0x7f6e401f9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:16:12.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.867+0000 7f6e45e21640 1 --2- 172.21.15.87:0/2024900568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e400cfaa0 0x7f6e400b66a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800ea40 tx=0x7f6e3800ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:12.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.867+0000 7f6e35ffb640 1 -- 172.21.15.87:0/2024900568 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3800ce40 con 0x7f6e400cfaa0 2024-09-17T03:16:12.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.867+0000 7f6e35ffb640 1 -- 172.21.15.87:0/2024900568 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e38004c10 con 0x7f6e400cfaa0 2024-09-17T03:16:12.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.867+0000 7f6e35ffb640 1 -- 172.21.15.87:0/2024900568 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e38005020 con 0x7f6e400cfaa0 2024-09-17T03:16:12.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.867+0000 7f6e47884640 1 -- 172.21.15.87:0/2024900568 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e401f6170 con 0x7f6e400cfaa0 2024-09-17T03:16:12.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.867+0000 7f6e47884640 1 -- 172.21.15.87:0/2024900568 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e401f6730 con 0x7f6e400cfaa0 2024-09-17T03:16:12.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.868+0000 7f6e47884640 1 -- 172.21.15.87:0/2024900568 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6e40160390 con 0x7f6e400cfaa0 2024-09-17T03:16:12.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.869+0000 7f6e35ffb640 1 -- 172.21.15.87:0/2024900568 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6e380051c0 con 0x7f6e400cfaa0 2024-09-17T03:16:12.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.869+0000 7f6e35ffb640 1 --2- 172.21.15.87:0/2024900568 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6e24078b90 0x7f6e2407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:12.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.869+0000 7f6e35ffb640 1 -- 172.21.15.87:0/2024900568 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f6e3809b830 con 0x7f6e400cfaa0 2024-09-17T03:16:12.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.869+0000 7f6e35ffb640 1 --2- 172.21.15.87:0/2024900568 >> [v2:172.21.15.87:6824/2430409749,v1:172.21.15.87:6825/2430409749] conn(0x7f6e2407eaa0 0x7f6e24080ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:12.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.869+0000 7f6e35ffb640 1 -- 172.21.15.87:0/2024900568 --> [v2:172.21.15.87:6824/2430409749,v1:172.21.15.87:6825/2430409749] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6e3801fe80 con 0x7f6e2407eaa0 2024-09-17T03:16:12.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.869+0000 7f6e35ffb640 1 -- 172.21.15.87:0/2024900568 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f6e380a02c0 con 0x7f6e400cfaa0 2024-09-17T03:16:12.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.869+0000 7f6e45620640 1 --2- 172.21.15.87:0/2024900568 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6e24078b90 0x7f6e2407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:12.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.869+0000 7f6e44e1f640 1 --2- 172.21.15.87:0/2024900568 >> [v2:172.21.15.87:6824/2430409749,v1:172.21.15.87:6825/2430409749] conn(0x7f6e2407eaa0 0x7f6e24080ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:12.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.870+0000 7f6e44e1f640 1 --2- 172.21.15.87:0/2024900568 >> [v2:172.21.15.87:6824/2430409749,v1:172.21.15.87:6825/2430409749] conn(0x7f6e2407eaa0 0x7f6e24080ef0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:12.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.870+0000 7f6e45620640 1 --2- 172.21.15.87:0/2024900568 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6e24078b90 0x7f6e2407b0a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c034000 tx=0x7f6e3c004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:12.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.870+0000 7f6e35ffb640 1 -- 172.21.15.87:0/2024900568 <== osd.0 v2:172.21.15.87:6824/2430409749 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6e3801fe80 con 0x7f6e2407eaa0 2024-09-17T03:16:12.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.883+0000 7f6e47884640 1 -- 172.21.15.87:0/2024900568 --> [v2:172.21.15.87:6824/2430409749,v1:172.21.15.87:6825/2430409749] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6e400d5250 con 0x7f6e2407eaa0 2024-09-17T03:16:12.883 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.883+0000 7f6e35ffb640 1 -- 172.21.15.87:0/2024900568 <== osd.0 v2:172.21.15.87:6824/2430409749 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6e400d5250 con 0x7f6e2407eaa0 2024-09-17T03:16:12.883 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-17T03:16:12.883 INFO:teuthology.orchestra.run.smithi087.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T03:16:12.884 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-17T03:16:12.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.883+0000 7f6e47884640 1 -- 172.21.15.87:0/2024900568 >> [v2:172.21.15.87:6824/2430409749,v1:172.21.15.87:6825/2430409749] conn(0x7f6e2407eaa0 msgr2=0x7f6e24080ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:12.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.883+0000 7f6e47884640 1 --2- 172.21.15.87:0/2024900568 >> [v2:172.21.15.87:6824/2430409749,v1:172.21.15.87:6825/2430409749] conn(0x7f6e2407eaa0 0x7f6e24080ef0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:12.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.883+0000 7f6e47884640 1 -- 172.21.15.87:0/2024900568 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6e24078b90 msgr2=0x7f6e2407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:12.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.883+0000 7f6e47884640 1 --2- 172.21.15.87:0/2024900568 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6e24078b90 0x7f6e2407b0a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c034000 tx=0x7f6e3c004730 comp rx=0 tx=0).stop 2024-09-17T03:16:12.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.883+0000 7f6e47884640 1 -- 172.21.15.87:0/2024900568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e400cfaa0 msgr2=0x7f6e400b66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:12.885 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.883+0000 7f6e47884640 1 --2- 172.21.15.87:0/2024900568 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6e400cfaa0 0x7f6e400b66a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6e3800ea40 tx=0x7f6e3800ef00 comp rx=0 tx=0).stop 2024-09-17T03:16:12.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.884+0000 7f6e45e21640 1 -- 172.21.15.87:0/2024900568 reap_dead start 2024-09-17T03:16:12.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.884+0000 7f6e47884640 1 -- 172.21.15.87:0/2024900568 shutdown_connections 2024-09-17T03:16:12.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.884+0000 7f6e47884640 1 -- 172.21.15.87:0/2024900568 >> 172.21.15.87:0/2024900568 conn(0x7f6e400d4510 msgr2=0x7f6e40160aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:12.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.884+0000 7f6e47884640 1 -- 172.21.15.87:0/2024900568 shutdown_connections 2024-09-17T03:16:12.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:12.884+0000 7f6e47884640 1 -- 172.21.15.87:0/2024900568 wait complete. 2024-09-17T03:16:12.894 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-17T03:16:13.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.008+0000 7f51b8539640 1 -- 172.21.15.87:0/1633601342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51b0070ef0 msgr2=0x7f51b00cf680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:13.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.008+0000 7f51b8539640 1 --2- 172.21.15.87:0/1633601342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51b0070ef0 0x7f51b00cf680 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f51a8005fd0 tx=0x7f51a802a8b0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.008+0000 7f51b8539640 1 -- 172.21.15.87:0/1633601342 shutdown_connections 2024-09-17T03:16:13.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.008+0000 7f51b8539640 1 --2- 172.21.15.87:0/1633601342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51b0070ef0 0x7f51b00cf680 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.008+0000 7f51b8539640 1 -- 172.21.15.87:0/1633601342 >> 172.21.15.87:0/1633601342 conn(0x7f51b006cfc0 msgr2=0x7f51b006d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:13.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.008+0000 7f51b8539640 1 -- 172.21.15.87:0/1633601342 shutdown_connections 2024-09-17T03:16:13.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.008+0000 7f51b8539640 1 -- 172.21.15.87:0/1633601342 wait complete. 2024-09-17T03:16:13.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.010+0000 7f51b8539640 1 Processor -- start 2024-09-17T03:16:13.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.010+0000 7f51b8539640 1 -- start start 2024-09-17T03:16:13.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.011+0000 7f51b8539640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f51b0070ef0 0x7f51b00ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:13.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.011+0000 7f51b8539640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b015fb40 0x7f51b0158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:13.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.011+0000 7f51b8539640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51b0158e40 0x7f51b00b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:13.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.011+0000 7f51b8539640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f51b00b3ef0 con 0x7f51b0158e40 2024-09-17T03:16:13.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.011+0000 7f51b8539640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f51b015b940 con 0x7f51b015fb40 2024-09-17T03:16:13.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.011+0000 7f51b8539640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f51b00b4070 con 0x7f51b0070ef0 2024-09-17T03:16:13.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.011+0000 7f51b6ad6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51b0158e40 0x7f51b00b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:13.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.011+0000 7f51b6ad6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51b0158e40 0x7f51b00b03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46582/0 (socket says 172.21.15.87:46582) 2024-09-17T03:16:13.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.011+0000 7f51b6ad6640 1 -- 172.21.15.87:0/3970564172 learned_addr learned my addr 172.21.15.87:0/3970564172 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:13.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.011+0000 7f51b5ad4640 1 --2- 172.21.15.87:0/3970564172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b015fb40 0x7f51b0158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:13.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.011+0000 7f51b62d5640 1 --2- 172.21.15.87:0/3970564172 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f51b0070ef0 0x7f51b00ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:13.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.012+0000 7f51b6ad6640 1 -- 172.21.15.87:0/3970564172 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f51b0070ef0 msgr2=0x7f51b00ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:13.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.012+0000 7f51b6ad6640 1 --2- 172.21.15.87:0/3970564172 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f51b0070ef0 0x7f51b00ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.012+0000 7f51b6ad6640 1 -- 172.21.15.87:0/3970564172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b015fb40 msgr2=0x7f51b0158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:13.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.012+0000 7f51b6ad6640 1 --2- 172.21.15.87:0/3970564172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b015fb40 0x7f51b0158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.012+0000 7f51b6ad6640 1 -- 172.21.15.87:0/3970564172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51b01f5850 con 0x7f51b0158e40 2024-09-17T03:16:13.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.012+0000 7f51b6ad6640 1 --2- 172.21.15.87:0/3970564172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51b0158e40 0x7f51b00b03e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f51a8009cd0 tx=0x7f51a8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:13.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.012+0000 7f51a6ffd640 1 -- 172.21.15.87:0/3970564172 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8031da0 con 0x7f51b0158e40 2024-09-17T03:16:13.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.012+0000 7f51a6ffd640 1 -- 172.21.15.87:0/3970564172 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51a8003e10 con 0x7f51b0158e40 2024-09-17T03:16:13.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.012+0000 7f51b8539640 1 -- 172.21.15.87:0/3970564172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51b01f5b30 con 0x7f51b0158e40 2024-09-17T03:16:13.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.012+0000 7f51a6ffd640 1 -- 172.21.15.87:0/3970564172 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a8033330 con 0x7f51b0158e40 2024-09-17T03:16:13.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.012+0000 7f51b8539640 1 -- 172.21.15.87:0/3970564172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51b00b1350 con 0x7f51b0158e40 2024-09-17T03:16:13.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.013+0000 7f51b8539640 1 -- 172.21.15.87:0/3970564172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51b00cf680 con 0x7f51b0158e40 2024-09-17T03:16:13.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.014+0000 7f51a6ffd640 1 -- 172.21.15.87:0/3970564172 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f51a80334d0 con 0x7f51b0158e40 2024-09-17T03:16:13.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.015+0000 7f51a6ffd640 1 --2- 172.21.15.87:0/3970564172 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5194078b90 0x7f519407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:13.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.015+0000 7f51a6ffd640 1 -- 172.21.15.87:0/3970564172 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f51a80bbfc0 con 0x7f51b0158e40 2024-09-17T03:16:13.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.015+0000 7f51b62d5640 1 --2- 172.21.15.87:0/3970564172 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5194078b90 0x7f519407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:13.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.015+0000 7f51b62d5640 1 --2- 172.21.15.87:0/3970564172 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5194078b90 0x7f519407b0a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f51ac007fd0 tx=0x7f51ac009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:13.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.019+0000 7f51a6ffd640 1 -- 172.21.15.87:0/3970564172 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f51a803f030 con 0x7f51b0158e40 2024-09-17T03:16:13.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.194+0000 7f51b8539640 1 -- 172.21.15.87:0/3970564172 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f51b00b15f0 con 0x7f5194078b90 2024-09-17T03:16:13.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.195+0000 7f51a6ffd640 1 -- 172.21.15.87:0/3970564172 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f51b00b15f0 con 0x7f5194078b90 2024-09-17T03:16:13.195 INFO:teuthology.orchestra.run.smithi087.stderr:instructed osd(s) 0 to deep-scrub 2024-09-17T03:16:13.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.197+0000 7f51b8539640 1 -- 172.21.15.87:0/3970564172 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5194078b90 msgr2=0x7f519407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:13.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.197+0000 7f51b8539640 1 --2- 172.21.15.87:0/3970564172 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5194078b90 0x7f519407b0a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f51ac007fd0 tx=0x7f51ac009040 comp rx=0 tx=0).stop 2024-09-17T03:16:13.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.197+0000 7f51b8539640 1 -- 172.21.15.87:0/3970564172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51b0158e40 msgr2=0x7f51b00b03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:13.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.197+0000 7f51b8539640 1 --2- 172.21.15.87:0/3970564172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51b0158e40 0x7f51b00b03e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f51a8009cd0 tx=0x7f51a8003c20 comp rx=0 tx=0).stop 2024-09-17T03:16:13.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.197+0000 7f51b8539640 1 -- 172.21.15.87:0/3970564172 shutdown_connections 2024-09-17T03:16:13.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.197+0000 7f51b8539640 1 --2- 172.21.15.87:0/3970564172 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f5194078b90 0x7f519407b0a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.197+0000 7f51b8539640 1 --2- 172.21.15.87:0/3970564172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51b0158e40 0x7f51b00b03e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.197+0000 7f51b8539640 1 --2- 172.21.15.87:0/3970564172 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f51b015fb40 0x7f51b0158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.197+0000 7f51b8539640 1 --2- 172.21.15.87:0/3970564172 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f51b0070ef0 0x7f51b00ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.197+0000 7f51b8539640 1 -- 172.21.15.87:0/3970564172 >> 172.21.15.87:0/3970564172 conn(0x7f51b006cfc0 msgr2=0x7f51b015a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:13.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.197+0000 7f51b8539640 1 -- 172.21.15.87:0/3970564172 shutdown_connections 2024-09-17T03:16:13.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.197+0000 7f51b8539640 1 -- 172.21.15.87:0/3970564172 wait complete. 2024-09-17T03:16:13.208 INFO:tasks.ceph:Scrubbing osd.1 2024-09-17T03:16:13.209 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-17T03:16:13.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.326+0000 7f0476020640 1 -- 172.21.15.87:0/840075717 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0458004f20 msgr2=0x7f045800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:13.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.326+0000 7f0476020640 1 --2- 172.21.15.87:0/840075717 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0458004f20 0x7f045800b9c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0468009f60 tx=0x7f0458005830 comp rx=0 tx=0).stop 2024-09-17T03:16:13.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.326+0000 7f046ffff640 1 -- 172.21.15.87:0/840075717 reap_dead start 2024-09-17T03:16:13.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.326+0000 7f0476020640 1 -- 172.21.15.87:0/840075717 shutdown_connections 2024-09-17T03:16:13.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.326+0000 7f0476020640 1 -- 172.21.15.87:0/840075717 >> 172.21.15.87:0/840075717 conn(0x7f0470069120 msgr2=0x7f0470069560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:13.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.326+0000 7f0476020640 1 -- 172.21.15.87:0/840075717 shutdown_connections 2024-09-17T03:16:13.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.326+0000 7f0476020640 1 -- 172.21.15.87:0/840075717 wait complete. 2024-09-17T03:16:13.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.328+0000 7f0476020640 1 Processor -- start 2024-09-17T03:16:13.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.328+0000 7f0476020640 1 -- start start 2024-09-17T03:16:13.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.329+0000 7f0476020640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0458004f20 0x7f04700b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:13.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.329+0000 7f0476020640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04701f68b0 0x7f04700c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:13.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.329+0000 7f0476020640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04700c7f10 0x7f04701f3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:13.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.329+0000 7f0476020640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0470160b60 con 0x7f04700c7f10 2024-09-17T03:16:13.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.329+0000 7f0476020640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f04701609e0 con 0x7f04701f68b0 2024-09-17T03:16:13.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.329+0000 7f0476020640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0470152e40 con 0x7f0458004f20 2024-09-17T03:16:13.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.329+0000 7f046ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04700c7f10 0x7f04701f3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:13.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.329+0000 7f046ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04700c7f10 0x7f04701f3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46594/0 (socket says 172.21.15.87:46594) 2024-09-17T03:16:13.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.329+0000 7f046ffff640 1 -- 172.21.15.87:0/645958590 learned_addr learned my addr 172.21.15.87:0/645958590 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:13.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.329+0000 7f046effd640 1 --2- 172.21.15.87:0/645958590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04701f68b0 0x7f04700c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:13.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.329+0000 7f046f7fe640 1 --2- 172.21.15.87:0/645958590 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0458004f20 0x7f04700b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:13.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.330+0000 7f046ffff640 1 -- 172.21.15.87:0/645958590 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0458004f20 msgr2=0x7f04700b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:13.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.330+0000 7f046ffff640 1 --2- 172.21.15.87:0/645958590 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0458004f20 0x7f04700b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.330+0000 7f046ffff640 1 -- 172.21.15.87:0/645958590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04701f68b0 msgr2=0x7f04700c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:13.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.330+0000 7f046ffff640 1 --2- 172.21.15.87:0/645958590 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04701f68b0 0x7f04700c5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.330+0000 7f046ffff640 1 -- 172.21.15.87:0/645958590 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04701f7090 con 0x7f04700c7f10 2024-09-17T03:16:13.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.330+0000 7f046ffff640 1 --2- 172.21.15.87:0/645958590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04700c7f10 0x7f04701f3af0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f046800b140 tx=0x7f0468032a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:13.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.330+0000 7f0457fff640 1 -- 172.21.15.87:0/645958590 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468041030 con 0x7f04700c7f10 2024-09-17T03:16:13.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.330+0000 7f0457fff640 1 -- 172.21.15.87:0/645958590 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0468003e30 con 0x7f04700c7f10 2024-09-17T03:16:13.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.330+0000 7f0457fff640 1 -- 172.21.15.87:0/645958590 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468004040 con 0x7f04700c7f10 2024-09-17T03:16:13.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.330+0000 7f0476020640 1 -- 172.21.15.87:0/645958590 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04701f7370 con 0x7f04700c7f10 2024-09-17T03:16:13.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.330+0000 7f0476020640 1 -- 172.21.15.87:0/645958590 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04701f35f0 con 0x7f04700c7f10 2024-09-17T03:16:13.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.331+0000 7f0476020640 1 -- 172.21.15.87:0/645958590 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f04700d2620 con 0x7f04700c7f10 2024-09-17T03:16:13.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.332+0000 7f0457fff640 1 -- 172.21.15.87:0/645958590 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0468039040 con 0x7f04700c7f10 2024-09-17T03:16:13.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.332+0000 7f0457fff640 1 --2- 172.21.15.87:0/645958590 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f045c078b90 0x7f045c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:13.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.332+0000 7f0457fff640 1 -- 172.21.15.87:0/645958590 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f04680bbc40 con 0x7f04700c7f10 2024-09-17T03:16:13.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.333+0000 7f046f7fe640 1 --2- 172.21.15.87:0/645958590 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f045c078b90 0x7f045c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:13.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.333+0000 7f0457fff640 1 --2- 172.21.15.87:0/645958590 >> [v2:172.21.15.87:6808/3467605435,v1:172.21.15.87:6809/3467605435] conn(0x7f045c07eaa0 0x7f045c080ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:13.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.333+0000 7f0457fff640 1 -- 172.21.15.87:0/645958590 --> [v2:172.21.15.87:6808/3467605435,v1:172.21.15.87:6809/3467605435] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0468043ec0 con 0x7f045c07eaa0 2024-09-17T03:16:13.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.333+0000 7f0457fff640 1 -- 172.21.15.87:0/645958590 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f04680bc0d0 con 0x7f04700c7f10 2024-09-17T03:16:13.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.333+0000 7f046effd640 1 --2- 172.21.15.87:0/645958590 >> [v2:172.21.15.87:6808/3467605435,v1:172.21.15.87:6809/3467605435] conn(0x7f045c07eaa0 0x7f045c080ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:13.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.333+0000 7f046effd640 1 --2- 172.21.15.87:0/645958590 >> [v2:172.21.15.87:6808/3467605435,v1:172.21.15.87:6809/3467605435] conn(0x7f045c07eaa0 0x7f045c080ef0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:13.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.333+0000 7f046f7fe640 1 --2- 172.21.15.87:0/645958590 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f045c078b90 0x7f045c07b0a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0464000ff0 tx=0x7f0464006910 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:13.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.334+0000 7f0457fff640 1 -- 172.21.15.87:0/645958590 <== osd.1 v2:172.21.15.87:6808/3467605435 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0468043ec0 con 0x7f045c07eaa0 2024-09-17T03:16:13.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.346+0000 7f0476020640 1 -- 172.21.15.87:0/645958590 --> [v2:172.21.15.87:6808/3467605435,v1:172.21.15.87:6809/3467605435] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f04701f6e00 con 0x7f045c07eaa0 2024-09-17T03:16:13.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.346+0000 7f0457fff640 1 -- 172.21.15.87:0/645958590 <== osd.1 v2:172.21.15.87:6808/3467605435 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f04701f6e00 con 0x7f045c07eaa0 2024-09-17T03:16:13.346 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-17T03:16:13.347 INFO:teuthology.orchestra.run.smithi087.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T03:16:13.347 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-17T03:16:13.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.346+0000 7f0476020640 1 -- 172.21.15.87:0/645958590 >> [v2:172.21.15.87:6808/3467605435,v1:172.21.15.87:6809/3467605435] conn(0x7f045c07eaa0 msgr2=0x7f045c080ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:13.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.346+0000 7f0476020640 1 --2- 172.21.15.87:0/645958590 >> [v2:172.21.15.87:6808/3467605435,v1:172.21.15.87:6809/3467605435] conn(0x7f045c07eaa0 0x7f045c080ef0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.346+0000 7f0476020640 1 -- 172.21.15.87:0/645958590 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f045c078b90 msgr2=0x7f045c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:13.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.346+0000 7f0476020640 1 --2- 172.21.15.87:0/645958590 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f045c078b90 0x7f045c07b0a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0464000ff0 tx=0x7f0464006910 comp rx=0 tx=0).stop 2024-09-17T03:16:13.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.347+0000 7f0476020640 1 -- 172.21.15.87:0/645958590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04700c7f10 msgr2=0x7f04701f3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:13.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.347+0000 7f0476020640 1 --2- 172.21.15.87:0/645958590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04700c7f10 0x7f04701f3af0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f046800b140 tx=0x7f0468032a10 comp rx=0 tx=0).stop 2024-09-17T03:16:13.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.347+0000 7f046ffff640 1 -- 172.21.15.87:0/645958590 reap_dead start 2024-09-17T03:16:13.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.347+0000 7f0476020640 1 -- 172.21.15.87:0/645958590 shutdown_connections 2024-09-17T03:16:13.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.347+0000 7f0476020640 1 -- 172.21.15.87:0/645958590 >> 172.21.15.87:0/645958590 conn(0x7f0470069120 msgr2=0x7f0470069560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:13.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.347+0000 7f0476020640 1 -- 172.21.15.87:0/645958590 shutdown_connections 2024-09-17T03:16:13.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.347+0000 7f0476020640 1 -- 172.21.15.87:0/645958590 wait complete. 2024-09-17T03:16:13.357 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-17T03:16:13.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.471+0000 7fd232d19640 1 -- 172.21.15.87:0/2220425026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd22c15f4d0 msgr2=0x7fd22c074de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:13.471 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.471+0000 7fd232d19640 1 --2- 172.21.15.87:0/2220425026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd22c15f4d0 0x7fd22c074de0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fd22c0710f0 tx=0x7fd22402aa30 comp rx=0 tx=0).stop 2024-09-17T03:16:13.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.471+0000 7fd232d19640 1 -- 172.21.15.87:0/2220425026 shutdown_connections 2024-09-17T03:16:13.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.471+0000 7fd232d19640 1 --2- 172.21.15.87:0/2220425026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd22c15f4d0 0x7fd22c074de0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.471+0000 7fd232d19640 1 --2- 172.21.15.87:0/2220425026 >> v2:172.21.15.149:3300/0 conn(0x7fd22c15f0c0 0x7fd22c0748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.471+0000 7fd232d19640 1 --2- 172.21.15.87:0/2220425026 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd22c0cc590 0x7fd22c15c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.471+0000 7fd232d19640 1 -- 172.21.15.87:0/2220425026 >> 172.21.15.87:0/2220425026 conn(0x7fd22c06ee70 msgr2=0x7fd22c06f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:13.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.471+0000 7fd232d19640 1 -- 172.21.15.87:0/2220425026 shutdown_connections 2024-09-17T03:16:13.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.471+0000 7fd232d19640 1 -- 172.21.15.87:0/2220425026 wait complete. 2024-09-17T03:16:13.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.473+0000 7fd232d19640 1 Processor -- start 2024-09-17T03:16:13.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.473+0000 7fd232d19640 1 -- start start 2024-09-17T03:16:13.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.474+0000 7fd232d19640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd22c0cc590 0x7fd22c14aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:13.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.474+0000 7fd232d19640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd22c15f0c0 0x7fd22c156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:13.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.474+0000 7fd232d19640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd22c15f4d0 0x7fd22c1574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:13.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.474+0000 7fd232d19640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c0c4f00 con 0x7fd22c0cc590 2024-09-17T03:16:13.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.474+0000 7fd232d19640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c0c6b10 con 0x7fd22c15f0c0 2024-09-17T03:16:13.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.474+0000 7fd232d19640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd22c0c5080 con 0x7fd22c15f4d0 2024-09-17T03:16:13.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.474+0000 7fd230ab5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd22c0cc590 0x7fd22c14aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:13.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.474+0000 7fd230ab5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd22c0cc590 0x7fd22c14aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46598/0 (socket says 172.21.15.87:46598) 2024-09-17T03:16:13.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.474+0000 7fd230ab5640 1 -- 172.21.15.87:0/1914283634 learned_addr learned my addr 172.21.15.87:0/1914283634 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:13.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.474+0000 7fd223fff640 1 --2- 172.21.15.87:0/1914283634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd22c15f0c0 0x7fd22c156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:13.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.474+0000 7fd2312b6640 1 --2- 172.21.15.87:0/1914283634 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd22c15f4d0 0x7fd22c1574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:13.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.475+0000 7fd230ab5640 1 -- 172.21.15.87:0/1914283634 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd22c15f4d0 msgr2=0x7fd22c1574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:13.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.475+0000 7fd230ab5640 1 --2- 172.21.15.87:0/1914283634 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd22c15f4d0 0x7fd22c1574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.475+0000 7fd230ab5640 1 -- 172.21.15.87:0/1914283634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd22c15f0c0 msgr2=0x7fd22c156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:13.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.475+0000 7fd230ab5640 1 --2- 172.21.15.87:0/1914283634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd22c15f0c0 0x7fd22c156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.475+0000 7fd230ab5640 1 -- 172.21.15.87:0/1914283634 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd22c1557e0 con 0x7fd22c0cc590 2024-09-17T03:16:13.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.475+0000 7fd230ab5640 1 --2- 172.21.15.87:0/1914283634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd22c0cc590 0x7fd22c14aa10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd22800d750 tx=0x7fd22800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:13.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.475+0000 7fd2217fa640 1 -- 172.21.15.87:0/1914283634 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd228017070 con 0x7fd22c0cc590 2024-09-17T03:16:13.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.475+0000 7fd2217fa640 1 -- 172.21.15.87:0/1914283634 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd228009d20 con 0x7fd22c0cc590 2024-09-17T03:16:13.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.475+0000 7fd2217fa640 1 -- 172.21.15.87:0/1914283634 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22800de60 con 0x7fd22c0cc590 2024-09-17T03:16:13.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.475+0000 7fd232d19640 1 -- 172.21.15.87:0/1914283634 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd22c155ac0 con 0x7fd22c0cc590 2024-09-17T03:16:13.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.475+0000 7fd232d19640 1 -- 172.21.15.87:0/1914283634 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd22c156080 con 0x7fd22c0cc590 2024-09-17T03:16:13.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.476+0000 7fd232d19640 1 -- 172.21.15.87:0/1914283634 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd22c15c470 con 0x7fd22c0cc590 2024-09-17T03:16:13.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.477+0000 7fd2217fa640 1 -- 172.21.15.87:0/1914283634 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd228004280 con 0x7fd22c0cc590 2024-09-17T03:16:13.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.477+0000 7fd2217fa640 1 --2- 172.21.15.87:0/1914283634 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd218078bc0 0x7fd21807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:13.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.477+0000 7fd2217fa640 1 -- 172.21.15.87:0/1914283634 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7fd22809a9a0 con 0x7fd22c0cc590 2024-09-17T03:16:13.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.477+0000 7fd223fff640 1 --2- 172.21.15.87:0/1914283634 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd218078bc0 0x7fd21807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:13.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.478+0000 7fd223fff640 1 --2- 172.21.15.87:0/1914283634 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd218078bc0 0x7fd21807b0d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd22c1565b0 tx=0x7fd21c005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:13.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.482+0000 7fd2217fa640 1 -- 172.21.15.87:0/1914283634 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd228062b20 con 0x7fd22c0cc590 2024-09-17T03:16:13.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.656+0000 7fd232d19640 1 -- 172.21.15.87:0/1914283634 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fd22c0d00f0 con 0x7fd218078bc0 2024-09-17T03:16:13.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.657+0000 7fd2217fa640 1 -- 172.21.15.87:0/1914283634 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd22c0d00f0 con 0x7fd218078bc0 2024-09-17T03:16:13.657 INFO:teuthology.orchestra.run.smithi087.stderr:instructed osd(s) 1 to deep-scrub 2024-09-17T03:16:13.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.659+0000 7fd232d19640 1 -- 172.21.15.87:0/1914283634 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd218078bc0 msgr2=0x7fd21807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:13.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.659+0000 7fd232d19640 1 --2- 172.21.15.87:0/1914283634 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd218078bc0 0x7fd21807b0d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd22c1565b0 tx=0x7fd21c005f90 comp rx=0 tx=0).stop 2024-09-17T03:16:13.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.659+0000 7fd232d19640 1 -- 172.21.15.87:0/1914283634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd22c0cc590 msgr2=0x7fd22c14aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:13.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.659+0000 7fd232d19640 1 --2- 172.21.15.87:0/1914283634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd22c0cc590 0x7fd22c14aa10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd22800d750 tx=0x7fd22800dc10 comp rx=0 tx=0).stop 2024-09-17T03:16:13.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.659+0000 7fd232d19640 1 -- 172.21.15.87:0/1914283634 shutdown_connections 2024-09-17T03:16:13.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.659+0000 7fd232d19640 1 --2- 172.21.15.87:0/1914283634 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd218078bc0 0x7fd21807b0d0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.659+0000 7fd232d19640 1 --2- 172.21.15.87:0/1914283634 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd22c15f4d0 0x7fd22c1574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.659+0000 7fd232d19640 1 --2- 172.21.15.87:0/1914283634 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd22c15f0c0 0x7fd22c156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.659+0000 7fd232d19640 1 --2- 172.21.15.87:0/1914283634 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd22c0cc590 0x7fd22c14aa10 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.659+0000 7fd232d19640 1 -- 172.21.15.87:0/1914283634 >> 172.21.15.87:0/1914283634 conn(0x7fd22c06ee70 msgr2=0x7fd22c073f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:13.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.659+0000 7fd232d19640 1 -- 172.21.15.87:0/1914283634 shutdown_connections 2024-09-17T03:16:13.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.659+0000 7fd232d19640 1 -- 172.21.15.87:0/1914283634 wait complete. 2024-09-17T03:16:13.670 INFO:tasks.ceph:Scrubbing osd.2 2024-09-17T03:16:13.670 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-17T03:16:13.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.787+0000 7fc3d5c4d640 1 -- 172.21.15.87:0/1444985 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc3d00ce960 msgr2=0x7fc3d0074830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:13.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.787+0000 7fc3d5c4d640 1 --2- 172.21.15.87:0/1444985 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc3d00ce960 0x7fc3d0074830 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4009860 tx=0x7fc3c402a5a0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.787+0000 7fc3d5c4d640 1 -- 172.21.15.87:0/1444985 shutdown_connections 2024-09-17T03:16:13.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.787+0000 7fc3d5c4d640 1 --2- 172.21.15.87:0/1444985 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc3d00ce960 0x7fc3d0074830 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.787+0000 7fc3d5c4d640 1 -- 172.21.15.87:0/1444985 >> 172.21.15.87:0/1444985 conn(0x7fc3d00d1180 msgr2=0x7fc3d00d1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:13.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.788+0000 7fc3d5c4d640 1 -- 172.21.15.87:0/1444985 shutdown_connections 2024-09-17T03:16:13.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.788+0000 7fc3d5c4d640 1 -- 172.21.15.87:0/1444985 wait complete. 2024-09-17T03:16:13.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.789+0000 7fc3d5c4d640 1 Processor -- start 2024-09-17T03:16:13.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.789+0000 7fc3d5c4d640 1 -- start start 2024-09-17T03:16:13.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.791+0000 7fc3d5c4d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3d0074d70 0x7fc3d00b5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:13.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.791+0000 7fc3d5c4d640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc3d00ce960 0x7fc3d0142c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:13.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.791+0000 7fc3d5c4d640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3d015d8b0 0x7fc3d00cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:13.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.791+0000 7fc3d5c4d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d0145bd0 con 0x7fc3d0074d70 2024-09-17T03:16:13.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.791+0000 7fc3d5c4d640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d0145a50 con 0x7fc3d015d8b0 2024-09-17T03:16:13.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.791+0000 7fc3d5c4d640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc3d0145d50 con 0x7fc3d00ce960 2024-09-17T03:16:13.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.791+0000 7fc3cf7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3d0074d70 0x7fc3d00b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:13.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.791+0000 7fc3cf7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3d0074d70 0x7fc3d00b5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46604/0 (socket says 172.21.15.87:46604) 2024-09-17T03:16:13.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.791+0000 7fc3cf7fe640 1 -- 172.21.15.87:0/3818263464 learned_addr learned my addr 172.21.15.87:0/3818263464 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:13.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.791+0000 7fc3cffff640 1 --2- 172.21.15.87:0/3818263464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3d015d8b0 0x7fc3d00cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:13.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.791+0000 7fc3ceffd640 1 --2- 172.21.15.87:0/3818263464 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc3d00ce960 0x7fc3d0142c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:13.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.791+0000 7fc3cf7fe640 1 -- 172.21.15.87:0/3818263464 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc3d00ce960 msgr2=0x7fc3d0142c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:13.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.791+0000 7fc3cf7fe640 1 --2- 172.21.15.87:0/3818263464 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc3d00ce960 0x7fc3d0142c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.791+0000 7fc3cf7fe640 1 -- 172.21.15.87:0/3818263464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3d015d8b0 msgr2=0x7fc3d00cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:13.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.791+0000 7fc3cf7fe640 1 --2- 172.21.15.87:0/3818263464 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc3d015d8b0 0x7fc3d00cd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.791+0000 7fc3cf7fe640 1 -- 172.21.15.87:0/3818263464 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d00b46a0 con 0x7fc3d0074d70 2024-09-17T03:16:13.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.791+0000 7fc3cf7fe640 1 --2- 172.21.15.87:0/3818263464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3d0074d70 0x7fc3d00b5740 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc3c000c730 tx=0x7fc3c000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:13.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.791+0000 7fc3b7fff640 1 -- 172.21.15.87:0/3818263464 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c0016020 con 0x7fc3d0074d70 2024-09-17T03:16:13.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.791+0000 7fc3b7fff640 1 -- 172.21.15.87:0/3818263464 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3c000a950 con 0x7fc3d0074d70 2024-09-17T03:16:13.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.791+0000 7fc3d5c4d640 1 -- 172.21.15.87:0/3818263464 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d00b4980 con 0x7fc3d0074d70 2024-09-17T03:16:13.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.791+0000 7fc3b7fff640 1 -- 172.21.15.87:0/3818263464 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c000ada0 con 0x7fc3d0074d70 2024-09-17T03:16:13.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.791+0000 7fc3d5c4d640 1 -- 172.21.15.87:0/3818263464 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d00b4f40 con 0x7fc3d0074d70 2024-09-17T03:16:13.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.792+0000 7fc3d5c4d640 1 -- 172.21.15.87:0/3818263464 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc3d0074830 con 0x7fc3d0074d70 2024-09-17T03:16:13.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.793+0000 7fc3b7fff640 1 -- 172.21.15.87:0/3818263464 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc3c0019020 con 0x7fc3d0074d70 2024-09-17T03:16:13.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.794+0000 7fc3b7fff640 1 --2- 172.21.15.87:0/3818263464 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc3bc078b90 0x7fc3bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:13.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.794+0000 7fc3b7fff640 1 -- 172.21.15.87:0/3818263464 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7fc3c009b0f0 con 0x7fc3d0074d70 2024-09-17T03:16:13.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.794+0000 7fc3ceffd640 1 --2- 172.21.15.87:0/3818263464 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc3bc078b90 0x7fc3bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:13.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.794+0000 7fc3b7fff640 1 --2- 172.21.15.87:0/3818263464 >> [v2:172.21.15.87:6800/1450834766,v1:172.21.15.87:6801/1450834766] conn(0x7fc3bc07eaa0 0x7fc3bc080ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:13.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.794+0000 7fc3b7fff640 1 -- 172.21.15.87:0/3818263464 --> [v2:172.21.15.87:6800/1450834766,v1:172.21.15.87:6801/1450834766] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc3bc0815c0 con 0x7fc3bc07eaa0 2024-09-17T03:16:13.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.794+0000 7fc3b7fff640 1 -- 172.21.15.87:0/3818263464 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fc3c009b580 con 0x7fc3d0074d70 2024-09-17T03:16:13.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.794+0000 7fc3cffff640 1 --2- 172.21.15.87:0/3818263464 >> [v2:172.21.15.87:6800/1450834766,v1:172.21.15.87:6801/1450834766] conn(0x7fc3bc07eaa0 0x7fc3bc080ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:13.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.794+0000 7fc3ceffd640 1 --2- 172.21.15.87:0/3818263464 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc3bc078b90 0x7fc3bc07b0a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4034000 tx=0x7fc3c4004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:13.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.794+0000 7fc3cffff640 1 --2- 172.21.15.87:0/3818263464 >> [v2:172.21.15.87:6800/1450834766,v1:172.21.15.87:6801/1450834766] conn(0x7fc3bc07eaa0 0x7fc3bc080ef0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:13.798 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.795+0000 7fc3b7fff640 1 -- 172.21.15.87:0/3818263464 <== osd.2 v2:172.21.15.87:6800/1450834766 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc3bc0815c0 con 0x7fc3bc07eaa0 2024-09-17T03:16:13.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.807+0000 7fc3d5c4d640 1 -- 172.21.15.87:0/3818263464 --> [v2:172.21.15.87:6800/1450834766,v1:172.21.15.87:6801/1450834766] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc3d0160ed0 con 0x7fc3bc07eaa0 2024-09-17T03:16:13.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.808+0000 7fc3b7fff640 1 -- 172.21.15.87:0/3818263464 <== osd.2 v2:172.21.15.87:6800/1450834766 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc3d0160ed0 con 0x7fc3bc07eaa0 2024-09-17T03:16:13.808 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-17T03:16:13.808 INFO:teuthology.orchestra.run.smithi087.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T03:16:13.808 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-17T03:16:13.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.808+0000 7fc3d5c4d640 1 -- 172.21.15.87:0/3818263464 >> [v2:172.21.15.87:6800/1450834766,v1:172.21.15.87:6801/1450834766] conn(0x7fc3bc07eaa0 msgr2=0x7fc3bc080ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:13.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.808+0000 7fc3d5c4d640 1 --2- 172.21.15.87:0/3818263464 >> [v2:172.21.15.87:6800/1450834766,v1:172.21.15.87:6801/1450834766] conn(0x7fc3bc07eaa0 0x7fc3bc080ef0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.808+0000 7fc3d5c4d640 1 -- 172.21.15.87:0/3818263464 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc3bc078b90 msgr2=0x7fc3bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:13.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.808+0000 7fc3d5c4d640 1 --2- 172.21.15.87:0/3818263464 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc3bc078b90 0x7fc3bc07b0a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4034000 tx=0x7fc3c4004730 comp rx=0 tx=0).stop 2024-09-17T03:16:13.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.808+0000 7fc3d5c4d640 1 -- 172.21.15.87:0/3818263464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3d0074d70 msgr2=0x7fc3d00b5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:13.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.808+0000 7fc3d5c4d640 1 --2- 172.21.15.87:0/3818263464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3d0074d70 0x7fc3d00b5740 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fc3c000c730 tx=0x7fc3c000cbf0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.808+0000 7fc3cffff640 1 -- 172.21.15.87:0/3818263464 reap_dead start 2024-09-17T03:16:13.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.808+0000 7fc3d5c4d640 1 -- 172.21.15.87:0/3818263464 shutdown_connections 2024-09-17T03:16:13.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.808+0000 7fc3d5c4d640 1 -- 172.21.15.87:0/3818263464 >> 172.21.15.87:0/3818263464 conn(0x7fc3d00d1180 msgr2=0x7fc3d015b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:13.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.808+0000 7fc3d5c4d640 1 -- 172.21.15.87:0/3818263464 shutdown_connections 2024-09-17T03:16:13.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.808+0000 7fc3d5c4d640 1 -- 172.21.15.87:0/3818263464 wait complete. 2024-09-17T03:16:13.819 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-17T03:16:13.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.933+0000 7f04e5407640 1 -- 172.21.15.87:0/3738197235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04e00d1470 msgr2=0x7f04e01673c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:13.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.933+0000 7f04e5407640 1 --2- 172.21.15.87:0/3738197235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04e00d1470 0x7f04e01673c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f04d8005fd0 tx=0x7f04d802a8b0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.933+0000 7f04e5407640 1 -- 172.21.15.87:0/3738197235 shutdown_connections 2024-09-17T03:16:13.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.933+0000 7f04e5407640 1 --2- 172.21.15.87:0/3738197235 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04e00d1470 0x7f04e01673c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.933+0000 7f04e5407640 1 -- 172.21.15.87:0/3738197235 >> 172.21.15.87:0/3738197235 conn(0x7f04e0069360 msgr2=0x7f04e00ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:13.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.933+0000 7f04e5407640 1 -- 172.21.15.87:0/3738197235 shutdown_connections 2024-09-17T03:16:13.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.933+0000 7f04e5407640 1 -- 172.21.15.87:0/3738197235 wait complete. 2024-09-17T03:16:13.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.935+0000 7f04e5407640 1 Processor -- start 2024-09-17T03:16:13.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.935+0000 7f04e5407640 1 -- start start 2024-09-17T03:16:13.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.936+0000 7f04e5407640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f04e00b3d00 0x7f04e0153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:13.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.936+0000 7f04e5407640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04e00d1470 0x7f04e01f7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:13.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.936+0000 7f04e5407640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04e015c6c0 0x7f04e01f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:13.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.936+0000 7f04e5407640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f04e0167a80 con 0x7f04e00d1470 2024-09-17T03:16:13.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.936+0000 7f04e5407640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f04e0167900 con 0x7f04e015c6c0 2024-09-17T03:16:13.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.936+0000 7f04e5407640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f04e0167c00 con 0x7f04e00b3d00 2024-09-17T03:16:13.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.936+0000 7f04de7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04e00d1470 0x7f04e01f7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:13.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.936+0000 7f04de7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04e00d1470 0x7f04e01f7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46606/0 (socket says 172.21.15.87:46606) 2024-09-17T03:16:13.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.936+0000 7f04de7fc640 1 -- 172.21.15.87:0/1914524815 learned_addr learned my addr 172.21.15.87:0/1914524815 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:13.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.936+0000 7f04deffd640 1 --2- 172.21.15.87:0/1914524815 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f04e00b3d00 0x7f04e0153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:13.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.936+0000 7f04df7fe640 1 --2- 172.21.15.87:0/1914524815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04e015c6c0 0x7f04e01f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:13.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.936+0000 7f04de7fc640 1 -- 172.21.15.87:0/1914524815 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f04e00b3d00 msgr2=0x7f04e0153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:13.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.936+0000 7f04de7fc640 1 --2- 172.21.15.87:0/1914524815 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f04e00b3d00 0x7f04e0153e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.936+0000 7f04de7fc640 1 -- 172.21.15.87:0/1914524815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04e015c6c0 msgr2=0x7f04e01f76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:13.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.936+0000 7f04de7fc640 1 --2- 172.21.15.87:0/1914524815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04e015c6c0 0x7f04e01f76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:13.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.936+0000 7f04de7fc640 1 -- 172.21.15.87:0/1914524815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e015ccc0 con 0x7f04e00d1470 2024-09-17T03:16:13.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.936+0000 7f04de7fc640 1 --2- 172.21.15.87:0/1914524815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04e00d1470 0x7f04e01f7170 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f04d400c430 tx=0x7f04d400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:13.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.937+0000 7f04c77fe640 1 -- 172.21.15.87:0/1914524815 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4011790 con 0x7f04e00d1470 2024-09-17T03:16:13.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.937+0000 7f04e5407640 1 -- 172.21.15.87:0/1914524815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e015cfa0 con 0x7f04e00d1470 2024-09-17T03:16:13.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.937+0000 7f04c77fe640 1 -- 172.21.15.87:0/1914524815 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04d4017070 con 0x7f04e00d1470 2024-09-17T03:16:13.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.937+0000 7f04c77fe640 1 -- 172.21.15.87:0/1914524815 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04d4011cc0 con 0x7f04e00d1470 2024-09-17T03:16:13.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.937+0000 7f04e5407640 1 -- 172.21.15.87:0/1914524815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e00cb980 con 0x7f04e00d1470 2024-09-17T03:16:13.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.938+0000 7f04e5407640 1 -- 172.21.15.87:0/1914524815 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e0160260 con 0x7f04e00d1470 2024-09-17T03:16:13.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.939+0000 7f04c77fe640 1 -- 172.21.15.87:0/1914524815 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f04d401c460 con 0x7f04e00d1470 2024-09-17T03:16:13.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.939+0000 7f04c77fe640 1 --2- 172.21.15.87:0/1914524815 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f04cc078bc0 0x7f04cc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:13.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.939+0000 7f04c77fe640 1 -- 172.21.15.87:0/1914524815 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f04d409a790 con 0x7f04e00d1470 2024-09-17T03:16:13.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.939+0000 7f04deffd640 1 --2- 172.21.15.87:0/1914524815 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f04cc078bc0 0x7f04cc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:13.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.940+0000 7f04deffd640 1 --2- 172.21.15.87:0/1914524815 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f04cc078bc0 0x7f04cc07b0d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f04d0001140 tx=0x7f04d0009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:13.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:13.944+0000 7f04c77fe640 1 -- 172.21.15.87:0/1914524815 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f04d4062800 con 0x7f04e00d1470 2024-09-17T03:16:14.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.122+0000 7f04e5407640 1 -- 172.21.15.87:0/1914524815 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f04e00cef40 con 0x7f04cc078bc0 2024-09-17T03:16:14.123 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.123+0000 7f04c77fe640 1 -- 172.21.15.87:0/1914524815 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f04e00cef40 con 0x7f04cc078bc0 2024-09-17T03:16:14.123 INFO:teuthology.orchestra.run.smithi087.stderr:instructed osd(s) 2 to deep-scrub 2024-09-17T03:16:14.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.125+0000 7f04e5407640 1 -- 172.21.15.87:0/1914524815 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f04cc078bc0 msgr2=0x7f04cc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:14.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.125+0000 7f04e5407640 1 --2- 172.21.15.87:0/1914524815 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f04cc078bc0 0x7f04cc07b0d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f04d0001140 tx=0x7f04d0009040 comp rx=0 tx=0).stop 2024-09-17T03:16:14.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.125+0000 7f04e5407640 1 -- 172.21.15.87:0/1914524815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04e00d1470 msgr2=0x7f04e01f7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:14.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.125+0000 7f04e5407640 1 --2- 172.21.15.87:0/1914524815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04e00d1470 0x7f04e01f7170 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f04d400c430 tx=0x7f04d400c8f0 comp rx=0 tx=0).stop 2024-09-17T03:16:14.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.125+0000 7f04e5407640 1 -- 172.21.15.87:0/1914524815 shutdown_connections 2024-09-17T03:16:14.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.125+0000 7f04e5407640 1 --2- 172.21.15.87:0/1914524815 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f04cc078bc0 0x7f04cc07b0d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:14.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.125+0000 7f04e5407640 1 --2- 172.21.15.87:0/1914524815 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f04e015c6c0 0x7f04e01f76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:14.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.125+0000 7f04e5407640 1 --2- 172.21.15.87:0/1914524815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04e00d1470 0x7f04e01f7170 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:14.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.125+0000 7f04e5407640 1 --2- 172.21.15.87:0/1914524815 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f04e00b3d00 0x7f04e0153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:14.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.125+0000 7f04e5407640 1 -- 172.21.15.87:0/1914524815 >> 172.21.15.87:0/1914524815 conn(0x7f04e0069360 msgr2=0x7f04e00b5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:14.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.125+0000 7f04e5407640 1 -- 172.21.15.87:0/1914524815 shutdown_connections 2024-09-17T03:16:14.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.125+0000 7f04e5407640 1 -- 172.21.15.87:0/1914524815 wait complete. 2024-09-17T03:16:14.136 INFO:tasks.ceph:Scrubbing osd.3 2024-09-17T03:16:14.137 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-17T03:16:14.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.254+0000 7f6d343da640 1 -- 172.21.15.87:0/1711169643 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d14004000 msgr2=0x7f6d14004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:14.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.254+0000 7f6d343da640 1 --2- 172.21.15.87:0/1711169643 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d14004000 0x7f6d14004800 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6d20006fd0 tx=0x7f6d14005830 comp rx=0 tx=0).stop 2024-09-17T03:16:14.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.254+0000 7f6d32977640 1 -- 172.21.15.87:0/1711169643 reap_dead start 2024-09-17T03:16:14.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.254+0000 7f6d343da640 1 -- 172.21.15.87:0/1711169643 shutdown_connections 2024-09-17T03:16:14.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.254+0000 7f6d343da640 1 -- 172.21.15.87:0/1711169643 >> 172.21.15.87:0/1711169643 conn(0x7f6d2c069120 msgr2=0x7f6d2c069560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:14.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.254+0000 7f6d343da640 1 -- 172.21.15.87:0/1711169643 shutdown_connections 2024-09-17T03:16:14.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.254+0000 7f6d343da640 1 -- 172.21.15.87:0/1711169643 wait complete. 2024-09-17T03:16:14.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.256+0000 7f6d343da640 1 Processor -- start 2024-09-17T03:16:14.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.256+0000 7f6d343da640 1 -- start start 2024-09-17T03:16:14.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.257+0000 7f6d343da640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d14004000 0x7f6d2c0cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:14.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.257+0000 7f6d343da640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d2c157a30 0x7f6d2c0b3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:14.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.257+0000 7f6d343da640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6d2c0b37c0 0x7f6d2c150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:14.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.257+0000 7f6d343da640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c152fc0 con 0x7f6d2c157a30 2024-09-17T03:16:14.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.257+0000 7f6d343da640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c152e40 con 0x7f6d14004000 2024-09-17T03:16:14.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.257+0000 7f6d343da640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6d2c153140 con 0x7f6d2c0b37c0 2024-09-17T03:16:14.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.257+0000 7f6d31975640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d2c157a30 0x7f6d2c0b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:14.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.257+0000 7f6d31975640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d2c157a30 0x7f6d2c0b3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46614/0 (socket says 172.21.15.87:46614) 2024-09-17T03:16:14.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.257+0000 7f6d31975640 1 -- 172.21.15.87:0/349092841 learned_addr learned my addr 172.21.15.87:0/349092841 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:14.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.257+0000 7f6d32977640 1 --2- 172.21.15.87:0/349092841 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6d2c0b37c0 0x7f6d2c150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:14.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.257+0000 7f6d32176640 1 --2- 172.21.15.87:0/349092841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d14004000 0x7f6d2c0cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:14.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.257+0000 7f6d31975640 1 -- 172.21.15.87:0/349092841 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6d2c0b37c0 msgr2=0x7f6d2c150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:14.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.257+0000 7f6d31975640 1 --2- 172.21.15.87:0/349092841 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6d2c0b37c0 0x7f6d2c150e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:14.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.257+0000 7f6d31975640 1 -- 172.21.15.87:0/349092841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d14004000 msgr2=0x7f6d2c0cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:14.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.257+0000 7f6d31975640 1 --2- 172.21.15.87:0/349092841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d14004000 0x7f6d2c0cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:14.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.257+0000 7f6d31975640 1 -- 172.21.15.87:0/349092841 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d2c0b3bd0 con 0x7f6d2c157a30 2024-09-17T03:16:14.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.257+0000 7f6d32176640 1 --2- 172.21.15.87:0/349092841 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6d14004000 0x7f6d2c0cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:16:14.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.257+0000 7f6d32977640 1 --2- 172.21.15.87:0/349092841 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6d2c0b37c0 0x7f6d2c150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:16:14.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.258+0000 7f6d31975640 1 --2- 172.21.15.87:0/349092841 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d2c157a30 0x7f6d2c0b3280 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6d2002b0f0 tx=0x7f6d20006c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:14.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.258+0000 7f6d2affd640 1 -- 172.21.15.87:0/349092841 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d20031040 con 0x7f6d2c157a30 2024-09-17T03:16:14.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.258+0000 7f6d2affd640 1 -- 172.21.15.87:0/349092841 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d200035f0 con 0x7f6d2c157a30 2024-09-17T03:16:14.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.258+0000 7f6d343da640 1 -- 172.21.15.87:0/349092841 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d2c1f32d0 con 0x7f6d2c157a30 2024-09-17T03:16:14.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.258+0000 7f6d2affd640 1 -- 172.21.15.87:0/349092841 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d20003a20 con 0x7f6d2c157a30 2024-09-17T03:16:14.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.258+0000 7f6d343da640 1 -- 172.21.15.87:0/349092841 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d2c1f3890 con 0x7f6d2c157a30 2024-09-17T03:16:14.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.259+0000 7f6d343da640 1 -- 172.21.15.87:0/349092841 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6d2c1536a0 con 0x7f6d2c157a30 2024-09-17T03:16:14.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.260+0000 7f6d2affd640 1 -- 172.21.15.87:0/349092841 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6d20034070 con 0x7f6d2c157a30 2024-09-17T03:16:14.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.260+0000 7f6d2affd640 1 --2- 172.21.15.87:0/349092841 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6d18078bc0 0x7f6d1807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:14.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.260+0000 7f6d2affd640 1 -- 172.21.15.87:0/349092841 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f6d200bb430 con 0x7f6d2c157a30 2024-09-17T03:16:14.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.260+0000 7f6d32176640 1 --2- 172.21.15.87:0/349092841 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6d18078bc0 0x7f6d1807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:14.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.260+0000 7f6d2affd640 1 --2- 172.21.15.87:0/349092841 >> [v2:172.21.15.87:6816/4253161216,v1:172.21.15.87:6817/4253161216] conn(0x7f6d1807ead0 0x7f6d18080f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:14.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.260+0000 7f6d2affd640 1 -- 172.21.15.87:0/349092841 --> [v2:172.21.15.87:6816/4253161216,v1:172.21.15.87:6817/4253161216] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6d2004ee50 con 0x7f6d1807ead0 2024-09-17T03:16:14.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.260+0000 7f6d2affd640 1 -- 172.21.15.87:0/349092841 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f6d200bb8c0 con 0x7f6d2c157a30 2024-09-17T03:16:14.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.260+0000 7f6d32977640 1 --2- 172.21.15.87:0/349092841 >> [v2:172.21.15.87:6816/4253161216,v1:172.21.15.87:6817/4253161216] conn(0x7f6d1807ead0 0x7f6d18080f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:14.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.261+0000 7f6d32176640 1 --2- 172.21.15.87:0/349092841 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6d18078bc0 0x7f6d1807b0d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c000f80 tx=0x7f6d1c006ef0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:14.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.261+0000 7f6d32977640 1 --2- 172.21.15.87:0/349092841 >> [v2:172.21.15.87:6816/4253161216,v1:172.21.15.87:6817/4253161216] conn(0x7f6d1807ead0 0x7f6d18080f20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:14.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.261+0000 7f6d2affd640 1 -- 172.21.15.87:0/349092841 <== osd.3 v2:172.21.15.87:6816/4253161216 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6d2004ee50 con 0x7f6d1807ead0 2024-09-17T03:16:14.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.274+0000 7f6d343da640 1 -- 172.21.15.87:0/349092841 --> [v2:172.21.15.87:6816/4253161216,v1:172.21.15.87:6817/4253161216] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6d2c0d0b90 con 0x7f6d1807ead0 2024-09-17T03:16:14.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.274+0000 7f6d2affd640 1 -- 172.21.15.87:0/349092841 <== osd.3 v2:172.21.15.87:6816/4253161216 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6d2c0d0b90 con 0x7f6d1807ead0 2024-09-17T03:16:14.274 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-17T03:16:14.274 INFO:teuthology.orchestra.run.smithi087.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T03:16:14.274 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-17T03:16:14.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.274+0000 7f6d343da640 1 -- 172.21.15.87:0/349092841 >> [v2:172.21.15.87:6816/4253161216,v1:172.21.15.87:6817/4253161216] conn(0x7f6d1807ead0 msgr2=0x7f6d18080f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:14.275 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.274+0000 7f6d343da640 1 --2- 172.21.15.87:0/349092841 >> [v2:172.21.15.87:6816/4253161216,v1:172.21.15.87:6817/4253161216] conn(0x7f6d1807ead0 0x7f6d18080f20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:14.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.275+0000 7f6d343da640 1 -- 172.21.15.87:0/349092841 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6d18078bc0 msgr2=0x7f6d1807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:14.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.275+0000 7f6d343da640 1 --2- 172.21.15.87:0/349092841 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6d18078bc0 0x7f6d1807b0d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c000f80 tx=0x7f6d1c006ef0 comp rx=0 tx=0).stop 2024-09-17T03:16:14.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.275+0000 7f6d343da640 1 -- 172.21.15.87:0/349092841 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d2c157a30 msgr2=0x7f6d2c0b3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:14.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.275+0000 7f6d343da640 1 --2- 172.21.15.87:0/349092841 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d2c157a30 0x7f6d2c0b3280 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6d2002b0f0 tx=0x7f6d20006c60 comp rx=0 tx=0).stop 2024-09-17T03:16:14.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.275+0000 7f6d32977640 1 -- 172.21.15.87:0/349092841 reap_dead start 2024-09-17T03:16:14.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.275+0000 7f6d343da640 1 -- 172.21.15.87:0/349092841 shutdown_connections 2024-09-17T03:16:14.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.275+0000 7f6d343da640 1 -- 172.21.15.87:0/349092841 >> 172.21.15.87:0/349092841 conn(0x7f6d2c069120 msgr2=0x7f6d2c1526a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:14.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.275+0000 7f6d343da640 1 -- 172.21.15.87:0/349092841 shutdown_connections 2024-09-17T03:16:14.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.275+0000 7f6d343da640 1 -- 172.21.15.87:0/349092841 wait complete. 2024-09-17T03:16:14.285 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-17T03:16:14.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.398+0000 7f4da65f1640 1 -- 172.21.15.87:0/2258033408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4da00d0230 msgr2=0x7f4da00d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:14.398 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.398+0000 7f4da65f1640 1 --2- 172.21.15.87:0/2258033408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4da00d0230 0x7f4da00d0640 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4d98005fd0 tx=0x7f4d9802aa10 comp rx=0 tx=0).stop 2024-09-17T03:16:14.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.398+0000 7f4da65f1640 1 -- 172.21.15.87:0/2258033408 shutdown_connections 2024-09-17T03:16:14.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.398+0000 7f4da65f1640 1 --2- 172.21.15.87:0/2258033408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4da00d0230 0x7f4da00d0640 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:14.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.398+0000 7f4da65f1640 1 --2- 172.21.15.87:0/2258033408 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4da00b4470 0x7f4da015c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:14.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.398+0000 7f4da65f1640 1 -- 172.21.15.87:0/2258033408 >> 172.21.15.87:0/2258033408 conn(0x7f4da0069330 msgr2=0x7f4da00ced60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:14.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.398+0000 7f4da65f1640 1 -- 172.21.15.87:0/2258033408 shutdown_connections 2024-09-17T03:16:14.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.398+0000 7f4da65f1640 1 -- 172.21.15.87:0/2258033408 wait complete. 2024-09-17T03:16:14.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.400+0000 7f4da65f1640 1 Processor -- start 2024-09-17T03:16:14.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.400+0000 7f4da65f1640 1 -- start start 2024-09-17T03:16:14.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.401+0000 7f4da65f1640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4da00b7f10 0x7f4da014b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:14.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.401+0000 7f4da65f1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4da00d0230 0x7f4da00c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:14.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.401+0000 7f4da65f1640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4da00b3c00 0x7f4da00b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:14.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.401+0000 7f4da65f1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4da00d0b80 con 0x7f4da00d0230 2024-09-17T03:16:14.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.401+0000 7f4da65f1640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f4da00b7d20 con 0x7f4da00b7f10 2024-09-17T03:16:14.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.401+0000 7f4da65f1640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f4da00d0d00 con 0x7f4da00b3c00 2024-09-17T03:16:14.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.401+0000 7f4d9f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4da00d0230 0x7f4da00c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:14.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.401+0000 7f4d9f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4da00d0230 0x7f4da00c9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46642/0 (socket says 172.21.15.87:46642) 2024-09-17T03:16:14.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.401+0000 7f4d9f7fe640 1 -- 172.21.15.87:0/3151068000 learned_addr learned my addr 172.21.15.87:0/3151068000 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:14.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.402+0000 7f4da4b8e640 1 --2- 172.21.15.87:0/3151068000 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4da00b3c00 0x7f4da00b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:14.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.402+0000 7f4d9ffff640 1 --2- 172.21.15.87:0/3151068000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4da00b7f10 0x7f4da014b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:14.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.402+0000 7f4d9f7fe640 1 -- 172.21.15.87:0/3151068000 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4da00b3c00 msgr2=0x7f4da00b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:14.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.402+0000 7f4d9f7fe640 1 --2- 172.21.15.87:0/3151068000 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4da00b3c00 0x7f4da00b4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:14.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.402+0000 7f4d9f7fe640 1 -- 172.21.15.87:0/3151068000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4da00b7f10 msgr2=0x7f4da014b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:14.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.402+0000 7f4d9f7fe640 1 --2- 172.21.15.87:0/3151068000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4da00b7f10 0x7f4da014b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:14.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.402+0000 7f4d9f7fe640 1 -- 172.21.15.87:0/3151068000 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da01f2a30 con 0x7f4da00d0230 2024-09-17T03:16:14.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.402+0000 7f4da4b8e640 1 --2- 172.21.15.87:0/3151068000 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4da00b3c00 0x7f4da00b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:16:14.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.402+0000 7f4d9ffff640 1 --2- 172.21.15.87:0/3151068000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4da00b7f10 0x7f4da014b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:16:14.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.402+0000 7f4d9f7fe640 1 --2- 172.21.15.87:0/3151068000 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4da00d0230 0x7f4da00c9f90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400c970 tx=0x7f4d9400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:14.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.402+0000 7f4d9cff9640 1 -- 172.21.15.87:0/3151068000 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d94007bb0 con 0x7f4da00d0230 2024-09-17T03:16:14.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.402+0000 7f4d9cff9640 1 -- 172.21.15.87:0/3151068000 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d94007e40 con 0x7f4da00d0230 2024-09-17T03:16:14.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.402+0000 7f4d9cff9640 1 -- 172.21.15.87:0/3151068000 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9400f690 con 0x7f4da00d0230 2024-09-17T03:16:14.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.402+0000 7f4da65f1640 1 -- 172.21.15.87:0/3151068000 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da00b4550 con 0x7f4da00d0230 2024-09-17T03:16:14.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.402+0000 7f4da65f1640 1 -- 172.21.15.87:0/3151068000 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da00cac10 con 0x7f4da00d0230 2024-09-17T03:16:14.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.403+0000 7f4da65f1640 1 -- 172.21.15.87:0/3151068000 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da00d11d0 con 0x7f4da00d0230 2024-09-17T03:16:14.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.404+0000 7f4d9cff9640 1 -- 172.21.15.87:0/3151068000 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4d9400f830 con 0x7f4da00d0230 2024-09-17T03:16:14.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.404+0000 7f4d9cff9640 1 --2- 172.21.15.87:0/3151068000 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4d8c078b90 0x7f4d8c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:14.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.404+0000 7f4d9cff9640 1 -- 172.21.15.87:0/3151068000 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f4d9409b2a0 con 0x7f4da00d0230 2024-09-17T03:16:14.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.405+0000 7f4d9ffff640 1 --2- 172.21.15.87:0/3151068000 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4d8c078b90 0x7f4d8c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:14.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.405+0000 7f4d9ffff640 1 --2- 172.21.15.87:0/3151068000 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4d8c078b90 0x7f4d8c07b0a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4d90000f60 tx=0x7f4d90009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:14.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.409+0000 7f4d9cff9640 1 -- 172.21.15.87:0/3151068000 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f4d94014030 con 0x7f4da00d0230 2024-09-17T03:16:14.582 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.582+0000 7f4da65f1640 1 -- 172.21.15.87:0/3151068000 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f4da00ca6c0 con 0x7f4d8c078b90 2024-09-17T03:16:14.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.583+0000 7f4d9cff9640 1 -- 172.21.15.87:0/3151068000 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4da00ca6c0 con 0x7f4d8c078b90 2024-09-17T03:16:14.583 INFO:teuthology.orchestra.run.smithi087.stderr:instructed osd(s) 3 to deep-scrub 2024-09-17T03:16:14.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.585+0000 7f4da65f1640 1 -- 172.21.15.87:0/3151068000 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4d8c078b90 msgr2=0x7f4d8c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:14.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.585+0000 7f4da65f1640 1 --2- 172.21.15.87:0/3151068000 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4d8c078b90 0x7f4d8c07b0a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4d90000f60 tx=0x7f4d90009040 comp rx=0 tx=0).stop 2024-09-17T03:16:14.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.585+0000 7f4da65f1640 1 -- 172.21.15.87:0/3151068000 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4da00d0230 msgr2=0x7f4da00c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:14.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.585+0000 7f4da65f1640 1 --2- 172.21.15.87:0/3151068000 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4da00d0230 0x7f4da00c9f90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4d9400c970 tx=0x7f4d9400ce30 comp rx=0 tx=0).stop 2024-09-17T03:16:14.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.585+0000 7f4da65f1640 1 -- 172.21.15.87:0/3151068000 shutdown_connections 2024-09-17T03:16:14.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.585+0000 7f4da65f1640 1 --2- 172.21.15.87:0/3151068000 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f4d8c078b90 0x7f4d8c07b0a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:14.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.585+0000 7f4da65f1640 1 --2- 172.21.15.87:0/3151068000 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f4da00b3c00 0x7f4da00b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:14.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.585+0000 7f4da65f1640 1 --2- 172.21.15.87:0/3151068000 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4da00d0230 0x7f4da00c9f90 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:14.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.585+0000 7f4da65f1640 1 --2- 172.21.15.87:0/3151068000 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f4da00b7f10 0x7f4da014b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:14.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.585+0000 7f4da65f1640 1 -- 172.21.15.87:0/3151068000 >> 172.21.15.87:0/3151068000 conn(0x7f4da0069330 msgr2=0x7f4da00b63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:14.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.585+0000 7f4da65f1640 1 -- 172.21.15.87:0/3151068000 shutdown_connections 2024-09-17T03:16:14.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.585+0000 7f4da65f1640 1 -- 172.21.15.87:0/3151068000 wait complete. 2024-09-17T03:16:14.596 INFO:tasks.ceph:Scrubbing osd.4 2024-09-17T03:16:14.596 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-17T03:16:14.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.714+0000 7f6c818b4640 1 -- 172.21.15.87:0/2176344825 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c64003a30 msgr2=0x7f6c64003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:14.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.714+0000 7f6c818b4640 1 --2- 172.21.15.87:0/2176344825 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c64003a30 0x7f6c64003ea0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009b50 tx=0x7f6c64005830 comp rx=0 tx=0).stop 2024-09-17T03:16:14.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.714+0000 7f6c7b7fe640 1 -- 172.21.15.87:0/2176344825 reap_dead start 2024-09-17T03:16:14.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.714+0000 7f6c818b4640 1 -- 172.21.15.87:0/2176344825 shutdown_connections 2024-09-17T03:16:14.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.714+0000 7f6c818b4640 1 -- 172.21.15.87:0/2176344825 >> 172.21.15.87:0/2176344825 conn(0x7f6c7c09d7e0 msgr2=0x7f6c7c08bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:14.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.714+0000 7f6c818b4640 1 -- 172.21.15.87:0/2176344825 shutdown_connections 2024-09-17T03:16:14.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.714+0000 7f6c818b4640 1 -- 172.21.15.87:0/2176344825 wait complete. 2024-09-17T03:16:14.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.716+0000 7f6c818b4640 1 Processor -- start 2024-09-17T03:16:14.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.716+0000 7f6c818b4640 1 -- start start 2024-09-17T03:16:14.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.717+0000 7f6c818b4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c64003a30 0x7f6c7c0cde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:14.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.717+0000 7f6c818b4640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6c7c1528c0 0x7f6c7c1f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:14.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.717+0000 7f6c818b4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c7c152cd0 0x7f6c7c0c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:14.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.717+0000 7f6c818b4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c0b9970 con 0x7f6c64003a30 2024-09-17T03:16:14.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.717+0000 7f6c818b4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c0b97f0 con 0x7f6c7c152cd0 2024-09-17T03:16:14.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.717+0000 7f6c818b4640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f6c7c0d25d0 con 0x7f6c7c1528c0 2024-09-17T03:16:14.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.718+0000 7f6c7affd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c64003a30 0x7f6c7c0cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:14.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.718+0000 7f6c7affd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c64003a30 0x7f6c7c0cde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46664/0 (socket says 172.21.15.87:46664) 2024-09-17T03:16:14.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.718+0000 7f6c7affd640 1 -- 172.21.15.87:0/2460082082 learned_addr learned my addr 172.21.15.87:0/2460082082 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:14.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.718+0000 7f6c7a7fc640 1 --2- 172.21.15.87:0/2460082082 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6c7c1528c0 0x7f6c7c1f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:14.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.718+0000 7f6c7b7fe640 1 --2- 172.21.15.87:0/2460082082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c7c152cd0 0x7f6c7c0c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:14.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.718+0000 7f6c7affd640 1 -- 172.21.15.87:0/2460082082 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6c7c1528c0 msgr2=0x7f6c7c1f53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:14.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.718+0000 7f6c7affd640 1 --2- 172.21.15.87:0/2460082082 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f6c7c1528c0 0x7f6c7c1f53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:14.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.718+0000 7f6c7affd640 1 -- 172.21.15.87:0/2460082082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c7c152cd0 msgr2=0x7f6c7c0c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:14.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.718+0000 7f6c7affd640 1 --2- 172.21.15.87:0/2460082082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c7c152cd0 0x7f6c7c0c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:14.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.718+0000 7f6c7affd640 1 -- 172.21.15.87:0/2460082082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7c1f6160 con 0x7f6c64003a30 2024-09-17T03:16:14.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.718+0000 7f6c7b7fe640 1 --2- 172.21.15.87:0/2460082082 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f6c7c152cd0 0x7f6c7c0c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:16:14.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.718+0000 7f6c7affd640 1 --2- 172.21.15.87:0/2460082082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c64003a30 0x7f6c7c0cde00 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009c70 tx=0x7f6c6c033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:14.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.718+0000 7f6c63fff640 1 -- 172.21.15.87:0/2460082082 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c038040 con 0x7f6c64003a30 2024-09-17T03:16:14.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.718+0000 7f6c63fff640 1 -- 172.21.15.87:0/2460082082 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c6c004020 con 0x7f6c64003a30 2024-09-17T03:16:14.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.718+0000 7f6c818b4640 1 -- 172.21.15.87:0/2460082082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7c0cd040 con 0x7f6c64003a30 2024-09-17T03:16:14.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.718+0000 7f6c63fff640 1 -- 172.21.15.87:0/2460082082 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c004470 con 0x7f6c64003a30 2024-09-17T03:16:14.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.718+0000 7f6c818b4640 1 -- 172.21.15.87:0/2460082082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c7c0cd600 con 0x7f6c64003a30 2024-09-17T03:16:14.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.719+0000 7f6c818b4640 1 -- 172.21.15.87:0/2460082082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6c7c0d07e0 con 0x7f6c64003a30 2024-09-17T03:16:14.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.720+0000 7f6c63fff640 1 -- 172.21.15.87:0/2460082082 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6c6c045020 con 0x7f6c64003a30 2024-09-17T03:16:14.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.721+0000 7f6c63fff640 1 --2- 172.21.15.87:0/2460082082 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6c68078bc0 0x7f6c6807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:14.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.721+0000 7f6c63fff640 1 -- 172.21.15.87:0/2460082082 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f6c6c0bbf90 con 0x7f6c64003a30 2024-09-17T03:16:14.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.721+0000 7f6c63fff640 1 --2- 172.21.15.87:0/2460082082 >> [v2:172.21.15.149:6802/2414907089,v1:172.21.15.149:6803/2414907089] conn(0x7f6c6807ead0 0x7f6c68080f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:14.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.721+0000 7f6c63fff640 1 -- 172.21.15.87:0/2460082082 --> [v2:172.21.15.149:6802/2414907089,v1:172.21.15.149:6803/2414907089] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6c680815f0 con 0x7f6c6807ead0 2024-09-17T03:16:14.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.721+0000 7f6c63fff640 1 -- 172.21.15.87:0/2460082082 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f6c6c0bc420 con 0x7f6c64003a30 2024-09-17T03:16:14.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.721+0000 7f6c7a7fc640 1 --2- 172.21.15.87:0/2460082082 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6c68078bc0 0x7f6c6807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:14.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.721+0000 7f6c7b7fe640 1 --2- 172.21.15.87:0/2460082082 >> [v2:172.21.15.149:6802/2414907089,v1:172.21.15.149:6803/2414907089] conn(0x7f6c6807ead0 0x7f6c68080f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:14.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.721+0000 7f6c7a7fc640 1 --2- 172.21.15.87:0/2460082082 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6c68078bc0 0x7f6c6807b0d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c1f2e00 tx=0x7f6c70006d90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:14.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.722+0000 7f6c7b7fe640 1 --2- 172.21.15.87:0/2460082082 >> [v2:172.21.15.149:6802/2414907089,v1:172.21.15.149:6803/2414907089] conn(0x7f6c6807ead0 0x7f6c68080f20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:14.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.722+0000 7f6c63fff640 1 -- 172.21.15.87:0/2460082082 <== osd.4 v2:172.21.15.149:6802/2414907089 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6c680815f0 con 0x7f6c6807ead0 2024-09-17T03:16:14.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.735+0000 7f6c818b4640 1 -- 172.21.15.87:0/2460082082 --> [v2:172.21.15.149:6802/2414907089,v1:172.21.15.149:6803/2414907089] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6c7c0d2b30 con 0x7f6c6807ead0 2024-09-17T03:16:14.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.735+0000 7f6c63fff640 1 -- 172.21.15.87:0/2460082082 <== osd.4 v2:172.21.15.149:6802/2414907089 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6c7c0d2b30 con 0x7f6c6807ead0 2024-09-17T03:16:14.735 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-17T03:16:14.735 INFO:teuthology.orchestra.run.smithi087.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T03:16:14.735 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-17T03:16:14.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.735+0000 7f6c818b4640 1 -- 172.21.15.87:0/2460082082 >> [v2:172.21.15.149:6802/2414907089,v1:172.21.15.149:6803/2414907089] conn(0x7f6c6807ead0 msgr2=0x7f6c68080f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:14.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.735+0000 7f6c818b4640 1 --2- 172.21.15.87:0/2460082082 >> [v2:172.21.15.149:6802/2414907089,v1:172.21.15.149:6803/2414907089] conn(0x7f6c6807ead0 0x7f6c68080f20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:14.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.735+0000 7f6c818b4640 1 -- 172.21.15.87:0/2460082082 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6c68078bc0 msgr2=0x7f6c6807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:14.736 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.735+0000 7f6c818b4640 1 --2- 172.21.15.87:0/2460082082 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f6c68078bc0 0x7f6c6807b0d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c1f2e00 tx=0x7f6c70006d90 comp rx=0 tx=0).stop 2024-09-17T03:16:14.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.735+0000 7f6c818b4640 1 -- 172.21.15.87:0/2460082082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c64003a30 msgr2=0x7f6c7c0cde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:14.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.735+0000 7f6c818b4640 1 --2- 172.21.15.87:0/2460082082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6c64003a30 0x7f6c7c0cde00 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009c70 tx=0x7f6c6c033670 comp rx=0 tx=0).stop 2024-09-17T03:16:14.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.735+0000 7f6c7b7fe640 1 -- 172.21.15.87:0/2460082082 reap_dead start 2024-09-17T03:16:14.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.735+0000 7f6c818b4640 1 -- 172.21.15.87:0/2460082082 shutdown_connections 2024-09-17T03:16:14.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.735+0000 7f6c818b4640 1 -- 172.21.15.87:0/2460082082 >> 172.21.15.87:0/2460082082 conn(0x7f6c7c09d7e0 msgr2=0x7f6c7c06eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:14.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.736+0000 7f6c818b4640 1 -- 172.21.15.87:0/2460082082 shutdown_connections 2024-09-17T03:16:14.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.736+0000 7f6c818b4640 1 -- 172.21.15.87:0/2460082082 wait complete. 2024-09-17T03:16:14.746 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-17T03:16:14.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.859+0000 7fc8f083f640 1 -- 172.21.15.87:0/3525229750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8ec0b76d0 msgr2=0x7fc8ec0cee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:14.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.859+0000 7fc8f083f640 1 --2- 172.21.15.87:0/3525229750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8ec0b76d0 0x7fc8ec0cee00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4005fd0 tx=0x7fc8e402a530 comp rx=0 tx=0).stop 2024-09-17T03:16:14.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.859+0000 7fc8f083f640 1 -- 172.21.15.87:0/3525229750 shutdown_connections 2024-09-17T03:16:14.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.859+0000 7fc8f083f640 1 --2- 172.21.15.87:0/3525229750 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc8ec0ccf80 0x7fc8ec0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:14.859 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.859+0000 7fc8f083f640 1 --2- 172.21.15.87:0/3525229750 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8ec0b76d0 0x7fc8ec0cee00 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:14.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.859+0000 7fc8f083f640 1 -- 172.21.15.87:0/3525229750 >> 172.21.15.87:0/3525229750 conn(0x7fc8ec0d2230 msgr2=0x7fc8ec0d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:14.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.859+0000 7fc8f083f640 1 -- 172.21.15.87:0/3525229750 shutdown_connections 2024-09-17T03:16:14.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.859+0000 7fc8f083f640 1 -- 172.21.15.87:0/3525229750 wait complete. 2024-09-17T03:16:14.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.861+0000 7fc8f083f640 1 Processor -- start 2024-09-17T03:16:14.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.861+0000 7fc8f083f640 1 -- start start 2024-09-17T03:16:14.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.862+0000 7fc8f083f640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc8ec0b76d0 0x7fc8ec1e50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:14.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.862+0000 7fc8f083f640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8ec0ccf80 0x7fc8ec09ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:14.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.862+0000 7fc8f083f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8ec0cf340 0x7fc8ec0b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:14.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.862+0000 7fc8f083f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec0d1250 con 0x7fc8ec0cf340 2024-09-17T03:16:14.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.862+0000 7fc8f083f640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec0b3100 con 0x7fc8ec0ccf80 2024-09-17T03:16:14.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.862+0000 7fc8f083f640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fc8ec0d13d0 con 0x7fc8ec0b76d0 2024-09-17T03:16:14.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.862+0000 7fc8ead9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8ec0cf340 0x7fc8ec0b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:14.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.862+0000 7fc8ead9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8ec0cf340 0x7fc8ec0b6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46676/0 (socket says 172.21.15.87:46676) 2024-09-17T03:16:14.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.862+0000 7fc8ead9d640 1 -- 172.21.15.87:0/1179088344 learned_addr learned my addr 172.21.15.87:0/1179088344 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:14.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.862+0000 7fc8ea59c640 1 --2- 172.21.15.87:0/1179088344 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc8ec0b76d0 0x7fc8ec1e50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:14.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.862+0000 7fc8ead9d640 1 -- 172.21.15.87:0/1179088344 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc8ec0b76d0 msgr2=0x7fc8ec1e50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:14.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.862+0000 7fc8e9d9b640 1 --2- 172.21.15.87:0/1179088344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8ec0ccf80 0x7fc8ec09ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:14.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.862+0000 7fc8ead9d640 1 --2- 172.21.15.87:0/1179088344 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc8ec0b76d0 0x7fc8ec1e50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:14.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.862+0000 7fc8ead9d640 1 -- 172.21.15.87:0/1179088344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8ec0ccf80 msgr2=0x7fc8ec09ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:14.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.862+0000 7fc8ead9d640 1 --2- 172.21.15.87:0/1179088344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8ec0ccf80 0x7fc8ec09ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:14.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.862+0000 7fc8ead9d640 1 -- 172.21.15.87:0/1179088344 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ec09c130 con 0x7fc8ec0cf340 2024-09-17T03:16:14.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.863+0000 7fc8ead9d640 1 --2- 172.21.15.87:0/1179088344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8ec0cf340 0x7fc8ec0b6310 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00d5f0 tx=0x7fc8dc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:14.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.863+0000 7fc8daffd640 1 -- 172.21.15.87:0/1179088344 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc017070 con 0x7fc8ec0cf340 2024-09-17T03:16:14.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.863+0000 7fc8daffd640 1 -- 172.21.15.87:0/1179088344 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8dc005330 con 0x7fc8ec0cf340 2024-09-17T03:16:14.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.863+0000 7fc8f083f640 1 -- 172.21.15.87:0/1179088344 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8ec09c3d0 con 0x7fc8ec0cf340 2024-09-17T03:16:14.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.863+0000 7fc8daffd640 1 -- 172.21.15.87:0/1179088344 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc0047a0 con 0x7fc8ec0cf340 2024-09-17T03:16:14.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.863+0000 7fc8f083f640 1 -- 172.21.15.87:0/1179088344 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8ec09c990 con 0x7fc8ec0cf340 2024-09-17T03:16:14.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.864+0000 7fc8f083f640 1 -- 172.21.15.87:0/1179088344 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8ec0cee00 con 0x7fc8ec0cf340 2024-09-17T03:16:14.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.865+0000 7fc8daffd640 1 -- 172.21.15.87:0/1179088344 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc8dc004940 con 0x7fc8ec0cf340 2024-09-17T03:16:14.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.865+0000 7fc8daffd640 1 --2- 172.21.15.87:0/1179088344 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc8cc078b90 0x7fc8cc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:14.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.866+0000 7fc8daffd640 1 -- 172.21.15.87:0/1179088344 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7fc8dc09aad0 con 0x7fc8ec0cf340 2024-09-17T03:16:14.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.866+0000 7fc8ea59c640 1 --2- 172.21.15.87:0/1179088344 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc8cc078b90 0x7fc8cc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:14.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.866+0000 7fc8ea59c640 1 --2- 172.21.15.87:0/1179088344 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc8cc078b90 0x7fc8cc07b0a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4033000 tx=0x7fc8e4004730 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:14.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:14.870+0000 7fc8daffd640 1 -- 172.21.15.87:0/1179088344 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fc8dc063220 con 0x7fc8ec0cf340 2024-09-17T03:16:15.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.043+0000 7fc8f083f640 1 -- 172.21.15.87:0/1179088344 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7fc8ec1e5640 con 0x7fc8cc078b90 2024-09-17T03:16:15.044 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.044+0000 7fc8daffd640 1 -- 172.21.15.87:0/1179088344 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc8ec1e5640 con 0x7fc8cc078b90 2024-09-17T03:16:15.044 INFO:teuthology.orchestra.run.smithi087.stderr:instructed osd(s) 4 to deep-scrub 2024-09-17T03:16:15.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.046+0000 7fc8f083f640 1 -- 172.21.15.87:0/1179088344 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc8cc078b90 msgr2=0x7fc8cc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:15.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.046+0000 7fc8f083f640 1 --2- 172.21.15.87:0/1179088344 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc8cc078b90 0x7fc8cc07b0a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4033000 tx=0x7fc8e4004730 comp rx=0 tx=0).stop 2024-09-17T03:16:15.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.046+0000 7fc8f083f640 1 -- 172.21.15.87:0/1179088344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8ec0cf340 msgr2=0x7fc8ec0b6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:15.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.046+0000 7fc8f083f640 1 --2- 172.21.15.87:0/1179088344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8ec0cf340 0x7fc8ec0b6310 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc00d5f0 tx=0x7fc8dc00dab0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.046+0000 7fc8f083f640 1 -- 172.21.15.87:0/1179088344 shutdown_connections 2024-09-17T03:16:15.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.046+0000 7fc8f083f640 1 --2- 172.21.15.87:0/1179088344 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fc8cc078b90 0x7fc8cc07b0a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.046+0000 7fc8f083f640 1 --2- 172.21.15.87:0/1179088344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8ec0cf340 0x7fc8ec0b6310 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.046+0000 7fc8f083f640 1 --2- 172.21.15.87:0/1179088344 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fc8ec0ccf80 0x7fc8ec09ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.046+0000 7fc8f083f640 1 --2- 172.21.15.87:0/1179088344 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fc8ec0b76d0 0x7fc8ec1e50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.046+0000 7fc8f083f640 1 -- 172.21.15.87:0/1179088344 >> 172.21.15.87:0/1179088344 conn(0x7fc8ec0d2230 msgr2=0x7fc8ec1e5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:15.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.046+0000 7fc8f083f640 1 -- 172.21.15.87:0/1179088344 shutdown_connections 2024-09-17T03:16:15.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.046+0000 7fc8f083f640 1 -- 172.21.15.87:0/1179088344 wait complete. 2024-09-17T03:16:15.057 INFO:tasks.ceph:Scrubbing osd.5 2024-09-17T03:16:15.057 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-17T03:16:15.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.174+0000 7fcf3a6da640 1 -- 172.21.15.87:0/2368637729 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf34070990 msgr2=0x7fcf340cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:15.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.174+0000 7fcf3a6da640 1 --2- 172.21.15.87:0/2368637729 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf34070990 0x7fcf340cf070 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fcf28005fd0 tx=0x7fcf2802a530 comp rx=0 tx=0).stop 2024-09-17T03:16:15.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.174+0000 7fcf3a6da640 1 -- 172.21.15.87:0/2368637729 shutdown_connections 2024-09-17T03:16:15.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.174+0000 7fcf3a6da640 1 --2- 172.21.15.87:0/2368637729 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf34070990 0x7fcf340cf070 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.174+0000 7fcf3a6da640 1 --2- 172.21.15.87:0/2368637729 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf3415b940 0x7fcf34070450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.174+0000 7fcf3a6da640 1 -- 172.21.15.87:0/2368637729 >> 172.21.15.87:0/2368637729 conn(0x7fcf340b2cc0 msgr2=0x7fcf340b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:15.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.174+0000 7fcf3a6da640 1 -- 172.21.15.87:0/2368637729 shutdown_connections 2024-09-17T03:16:15.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.174+0000 7fcf3a6da640 1 -- 172.21.15.87:0/2368637729 wait complete. 2024-09-17T03:16:15.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.176+0000 7fcf3a6da640 1 Processor -- start 2024-09-17T03:16:15.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.176+0000 7fcf3a6da640 1 -- start start 2024-09-17T03:16:15.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.177+0000 7fcf3a6da640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf34070990 0x7fcf3409cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:15.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.177+0000 7fcf3a6da640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf340cf5b0 0x7fcf3414d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:15.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.177+0000 7fcf3a6da640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf3415b940 0x7fcf340c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:15.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.177+0000 7fcf3a6da640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcf340b5490 con 0x7fcf340cf5b0 2024-09-17T03:16:15.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.177+0000 7fcf3a6da640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fcf34070190 con 0x7fcf34070990 2024-09-17T03:16:15.179 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.177+0000 7fcf3a6da640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fcf340b5610 con 0x7fcf3415b940 2024-09-17T03:16:15.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.177+0000 7fcf337fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf340cf5b0 0x7fcf3414d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:15.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.177+0000 7fcf337fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf340cf5b0 0x7fcf3414d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46702/0 (socket says 172.21.15.87:46702) 2024-09-17T03:16:15.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.177+0000 7fcf337fe640 1 -- 172.21.15.87:0/1730619041 learned_addr learned my addr 172.21.15.87:0/1730619041 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:15.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.177+0000 7fcf33fff640 1 --2- 172.21.15.87:0/1730619041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf34070990 0x7fcf3409cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:15.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.177+0000 7fcf38c77640 1 --2- 172.21.15.87:0/1730619041 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf3415b940 0x7fcf340c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:15.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.178+0000 7fcf337fe640 1 -- 172.21.15.87:0/1730619041 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf3415b940 msgr2=0x7fcf340c4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:15.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.178+0000 7fcf337fe640 1 --2- 172.21.15.87:0/1730619041 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fcf3415b940 0x7fcf340c4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.180 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.178+0000 7fcf337fe640 1 -- 172.21.15.87:0/1730619041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf34070990 msgr2=0x7fcf3409cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:15.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.178+0000 7fcf337fe640 1 --2- 172.21.15.87:0/1730619041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf34070990 0x7fcf3409cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.178+0000 7fcf337fe640 1 -- 172.21.15.87:0/1730619041 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf341f7920 con 0x7fcf340cf5b0 2024-09-17T03:16:15.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.178+0000 7fcf33fff640 1 --2- 172.21.15.87:0/1730619041 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fcf34070990 0x7fcf3409cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-17T03:16:15.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.178+0000 7fcf337fe640 1 --2- 172.21.15.87:0/1730619041 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf340cf5b0 0x7fcf3414d540 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fcf28006010 tx=0x7fcf280038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:15.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.178+0000 7fcf30ff9640 1 -- 172.21.15.87:0/1730619041 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2803e030 con 0x7fcf340cf5b0 2024-09-17T03:16:15.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.178+0000 7fcf30ff9640 1 -- 172.21.15.87:0/1730619041 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf28003d90 con 0x7fcf340cf5b0 2024-09-17T03:16:15.181 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.178+0000 7fcf3a6da640 1 -- 172.21.15.87:0/1730619041 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf341f6d10 con 0x7fcf340cf5b0 2024-09-17T03:16:15.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.178+0000 7fcf30ff9640 1 -- 172.21.15.87:0/1730619041 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28035040 con 0x7fcf340cf5b0 2024-09-17T03:16:15.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.178+0000 7fcf3a6da640 1 -- 172.21.15.87:0/1730619041 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf341f72d0 con 0x7fcf340cf5b0 2024-09-17T03:16:15.182 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.179+0000 7fcf3a6da640 1 -- 172.21.15.87:0/1730619041 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcf340b5ae0 con 0x7fcf340cf5b0 2024-09-17T03:16:15.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.180+0000 7fcf30ff9640 1 -- 172.21.15.87:0/1730619041 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fcf28041030 con 0x7fcf340cf5b0 2024-09-17T03:16:15.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.180+0000 7fcf30ff9640 1 --2- 172.21.15.87:0/1730619041 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcf20078b90 0x7fcf2007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:15.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.180+0000 7fcf30ff9640 1 -- 172.21.15.87:0/1730619041 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7fcf280bb880 con 0x7fcf340cf5b0 2024-09-17T03:16:15.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.180+0000 7fcf30ff9640 1 --2- 172.21.15.87:0/1730619041 >> [v2:172.21.15.149:6826/2182999530,v1:172.21.15.149:6827/2182999530] conn(0x7fcf2007eaa0 0x7fcf20080ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:15.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.180+0000 7fcf30ff9640 1 -- 172.21.15.87:0/1730619041 --> [v2:172.21.15.149:6826/2182999530,v1:172.21.15.149:6827/2182999530] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcf28005310 con 0x7fcf2007eaa0 2024-09-17T03:16:15.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.180+0000 7fcf30ff9640 1 -- 172.21.15.87:0/1730619041 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fcf280bbd10 con 0x7fcf340cf5b0 2024-09-17T03:16:15.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.180+0000 7fcf33fff640 1 --2- 172.21.15.87:0/1730619041 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcf20078b90 0x7fcf2007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:15.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.181+0000 7fcf38c77640 1 --2- 172.21.15.87:0/1730619041 >> [v2:172.21.15.149:6826/2182999530,v1:172.21.15.149:6827/2182999530] conn(0x7fcf2007eaa0 0x7fcf20080ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:15.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.181+0000 7fcf33fff640 1 --2- 172.21.15.87:0/1730619041 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcf20078b90 0x7fcf2007b0a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcf24005fd0 tx=0x7fcf2400f360 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:15.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.181+0000 7fcf38c77640 1 --2- 172.21.15.87:0/1730619041 >> [v2:172.21.15.149:6826/2182999530,v1:172.21.15.149:6827/2182999530] conn(0x7fcf2007eaa0 0x7fcf20080ef0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:15.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.182+0000 7fcf30ff9640 1 -- 172.21.15.87:0/1730619041 <== osd.5 v2:172.21.15.149:6826/2182999530 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcf28005310 con 0x7fcf2007eaa0 2024-09-17T03:16:15.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.194+0000 7fcf3a6da640 1 -- 172.21.15.87:0/1730619041 --> [v2:172.21.15.149:6826/2182999530,v1:172.21.15.149:6827/2182999530] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fcf341f6ac0 con 0x7fcf2007eaa0 2024-09-17T03:16:15.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.195+0000 7fcf30ff9640 1 -- 172.21.15.87:0/1730619041 <== osd.5 v2:172.21.15.149:6826/2182999530 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fcf341f6ac0 con 0x7fcf2007eaa0 2024-09-17T03:16:15.195 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-17T03:16:15.195 INFO:teuthology.orchestra.run.smithi087.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T03:16:15.195 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-17T03:16:15.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.195+0000 7fcf3a6da640 1 -- 172.21.15.87:0/1730619041 >> [v2:172.21.15.149:6826/2182999530,v1:172.21.15.149:6827/2182999530] conn(0x7fcf2007eaa0 msgr2=0x7fcf20080ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:15.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.195+0000 7fcf3a6da640 1 --2- 172.21.15.87:0/1730619041 >> [v2:172.21.15.149:6826/2182999530,v1:172.21.15.149:6827/2182999530] conn(0x7fcf2007eaa0 0x7fcf20080ef0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.195+0000 7fcf3a6da640 1 -- 172.21.15.87:0/1730619041 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcf20078b90 msgr2=0x7fcf2007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:15.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.195+0000 7fcf3a6da640 1 --2- 172.21.15.87:0/1730619041 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fcf20078b90 0x7fcf2007b0a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcf24005fd0 tx=0x7fcf2400f360 comp rx=0 tx=0).stop 2024-09-17T03:16:15.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.195+0000 7fcf3a6da640 1 -- 172.21.15.87:0/1730619041 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf340cf5b0 msgr2=0x7fcf3414d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:15.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.195+0000 7fcf3a6da640 1 --2- 172.21.15.87:0/1730619041 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf340cf5b0 0x7fcf3414d540 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fcf28006010 tx=0x7fcf280038a0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.195+0000 7fcf38c77640 1 -- 172.21.15.87:0/1730619041 reap_dead start 2024-09-17T03:16:15.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.195+0000 7fcf3a6da640 1 -- 172.21.15.87:0/1730619041 shutdown_connections 2024-09-17T03:16:15.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.195+0000 7fcf3a6da640 1 -- 172.21.15.87:0/1730619041 >> 172.21.15.87:0/1730619041 conn(0x7fcf340b2cc0 msgr2=0x7fcf340cdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:15.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.195+0000 7fcf3a6da640 1 -- 172.21.15.87:0/1730619041 shutdown_connections 2024-09-17T03:16:15.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.195+0000 7fcf3a6da640 1 -- 172.21.15.87:0/1730619041 wait complete. 2024-09-17T03:16:15.206 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-17T03:16:15.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.319+0000 7f45073ba640 1 -- 172.21.15.87:0/4239670180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45000d1080 msgr2=0x7f45000ccf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:15.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.319+0000 7f45073ba640 1 --2- 172.21.15.87:0/4239670180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45000d1080 0x7f45000ccf80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f45000747b0 tx=0x7f44f802aa10 comp rx=0 tx=0).stop 2024-09-17T03:16:15.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.319+0000 7f45073ba640 1 -- 172.21.15.87:0/4239670180 shutdown_connections 2024-09-17T03:16:15.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.319+0000 7f45073ba640 1 --2- 172.21.15.87:0/4239670180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45000d1080 0x7f45000ccf80 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.319+0000 7f45073ba640 1 --2- 172.21.15.87:0/4239670180 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f45000cf340 0x7f45000d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.319+0000 7f45073ba640 1 --2- 172.21.15.87:0/4239670180 >> v2:172.21.15.149:3300/0 conn(0x7f45000b76d0 0x7f45000cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.319+0000 7f45073ba640 1 -- 172.21.15.87:0/4239670180 >> 172.21.15.87:0/4239670180 conn(0x7f45000d2230 msgr2=0x7f45000d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:15.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.319+0000 7f45073ba640 1 -- 172.21.15.87:0/4239670180 shutdown_connections 2024-09-17T03:16:15.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.319+0000 7f45073ba640 1 -- 172.21.15.87:0/4239670180 wait complete. 2024-09-17T03:16:15.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.321+0000 7f45073ba640 1 Processor -- start 2024-09-17T03:16:15.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.321+0000 7f45073ba640 1 -- start start 2024-09-17T03:16:15.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.322+0000 7f45073ba640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45000b76d0 0x7f4500155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:15.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.322+0000 7f45073ba640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f45000cf340 0x7f450006def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:15.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.322+0000 7f45073ba640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45000d1080 0x7f450006e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:15.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.322+0000 7f45073ba640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f45000cd4c0 con 0x7f45000b76d0 2024-09-17T03:16:15.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.322+0000 7f45073ba640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f45000b30d0 con 0x7f45000d1080 2024-09-17T03:16:15.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.322+0000 7f45073ba640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f45000cd640 con 0x7f45000cf340 2024-09-17T03:16:15.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.323+0000 7f4505156640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45000b76d0 0x7f4500155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:15.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.323+0000 7f4505156640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45000b76d0 0x7f4500155f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48738/0 (socket says 172.21.15.87:48738) 2024-09-17T03:16:15.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.323+0000 7f4505156640 1 -- 172.21.15.87:0/2202233079 learned_addr learned my addr 172.21.15.87:0/2202233079 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:15.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.323+0000 7f4504955640 1 --2- 172.21.15.87:0/2202233079 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f45000cf340 0x7f450006def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:15.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.323+0000 7f4505957640 1 --2- 172.21.15.87:0/2202233079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45000d1080 0x7f450006e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:15.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.323+0000 7f4505156640 1 -- 172.21.15.87:0/2202233079 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f45000cf340 msgr2=0x7f450006def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:15.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.323+0000 7f4505156640 1 --2- 172.21.15.87:0/2202233079 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f45000cf340 0x7f450006def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.323+0000 7f4505156640 1 -- 172.21.15.87:0/2202233079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45000d1080 msgr2=0x7f450006e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:15.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.323+0000 7f4505156640 1 --2- 172.21.15.87:0/2202233079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45000d1080 0x7f450006e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.323+0000 7f4505156640 1 -- 172.21.15.87:0/2202233079 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45000cba30 con 0x7f45000b76d0 2024-09-17T03:16:15.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.323+0000 7f4505156640 1 --2- 172.21.15.87:0/2202233079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45000b76d0 0x7f4500155f50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00c940 tx=0x7f44fc00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:15.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.323+0000 7f44f5ffb640 1 -- 172.21.15.87:0/2202233079 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc007b50 con 0x7f45000b76d0 2024-09-17T03:16:15.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.323+0000 7f44f5ffb640 1 -- 172.21.15.87:0/2202233079 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44fc007de0 con 0x7f45000b76d0 2024-09-17T03:16:15.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.323+0000 7f45073ba640 1 -- 172.21.15.87:0/2202233079 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45000cbd10 con 0x7f45000b76d0 2024-09-17T03:16:15.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.323+0000 7f44f5ffb640 1 -- 172.21.15.87:0/2202233079 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc00f690 con 0x7f45000b76d0 2024-09-17T03:16:15.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.324+0000 7f45073ba640 1 -- 172.21.15.87:0/2202233079 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f450009f830 con 0x7f45000b76d0 2024-09-17T03:16:15.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.325+0000 7f45073ba640 1 -- 172.21.15.87:0/2202233079 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45000d0b40 con 0x7f45000b76d0 2024-09-17T03:16:15.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.325+0000 7f44f5ffb640 1 -- 172.21.15.87:0/2202233079 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f44fc00f830 con 0x7f45000b76d0 2024-09-17T03:16:15.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.325+0000 7f44f5ffb640 1 --2- 172.21.15.87:0/2202233079 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f44e4078bc0 0x7f44e407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:15.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.326+0000 7f44f5ffb640 1 -- 172.21.15.87:0/2202233079 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f44fc09af50 con 0x7f45000b76d0 2024-09-17T03:16:15.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.326+0000 7f4504955640 1 --2- 172.21.15.87:0/2202233079 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f44e4078bc0 0x7f44e407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:15.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.326+0000 7f4504955640 1 --2- 172.21.15.87:0/2202233079 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f44e4078bc0 0x7f44e407b0d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f45000ca560 tx=0x7f44f000a250 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:15.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.330+0000 7f44f5ffb640 1 -- 172.21.15.87:0/2202233079 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f44fc014030 con 0x7f45000b76d0 2024-09-17T03:16:15.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.508+0000 7f45073ba640 1 -- 172.21.15.87:0/2202233079 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f4500071200 con 0x7f44e4078bc0 2024-09-17T03:16:15.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.509+0000 7f44f5ffb640 1 -- 172.21.15.87:0/2202233079 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4500071200 con 0x7f44e4078bc0 2024-09-17T03:16:15.508 INFO:teuthology.orchestra.run.smithi087.stderr:instructed osd(s) 5 to deep-scrub 2024-09-17T03:16:15.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.511+0000 7f45073ba640 1 -- 172.21.15.87:0/2202233079 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f44e4078bc0 msgr2=0x7f44e407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:15.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.511+0000 7f45073ba640 1 --2- 172.21.15.87:0/2202233079 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f44e4078bc0 0x7f44e407b0d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f45000ca560 tx=0x7f44f000a250 comp rx=0 tx=0).stop 2024-09-17T03:16:15.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.511+0000 7f45073ba640 1 -- 172.21.15.87:0/2202233079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45000b76d0 msgr2=0x7f4500155f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:15.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.511+0000 7f45073ba640 1 --2- 172.21.15.87:0/2202233079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45000b76d0 0x7f4500155f50 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00c940 tx=0x7f44fc00ce00 comp rx=0 tx=0).stop 2024-09-17T03:16:15.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.511+0000 7f45073ba640 1 -- 172.21.15.87:0/2202233079 shutdown_connections 2024-09-17T03:16:15.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.511+0000 7f45073ba640 1 --2- 172.21.15.87:0/2202233079 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f44e4078bc0 0x7f44e407b0d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.511+0000 7f45073ba640 1 --2- 172.21.15.87:0/2202233079 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f45000d1080 0x7f450006e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.511+0000 7f45073ba640 1 --2- 172.21.15.87:0/2202233079 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f45000cf340 0x7f450006def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.511+0000 7f45073ba640 1 --2- 172.21.15.87:0/2202233079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45000b76d0 0x7f4500155f50 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.511+0000 7f45073ba640 1 -- 172.21.15.87:0/2202233079 >> 172.21.15.87:0/2202233079 conn(0x7f45000d2230 msgr2=0x7f4500072080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:15.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.511+0000 7f45073ba640 1 -- 172.21.15.87:0/2202233079 shutdown_connections 2024-09-17T03:16:15.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.511+0000 7f45073ba640 1 -- 172.21.15.87:0/2202233079 wait complete. 2024-09-17T03:16:15.522 INFO:tasks.ceph:Scrubbing osd.6 2024-09-17T03:16:15.522 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-17T03:16:15.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.640+0000 7f36968d5640 1 -- 172.21.15.87:0/101460547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3690160170 msgr2=0x7f3690160580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:15.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.640+0000 7f36968d5640 1 --2- 172.21.15.87:0/101460547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3690160170 0x7f3690160580 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f36900749f0 tx=0x7f368802aa30 comp rx=0 tx=0).stop 2024-09-17T03:16:15.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.640+0000 7f36968d5640 1 -- 172.21.15.87:0/101460547 shutdown_connections 2024-09-17T03:16:15.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.640+0000 7f36968d5640 1 --2- 172.21.15.87:0/101460547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3690160170 0x7f3690160580 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.640+0000 7f36968d5640 1 --2- 172.21.15.87:0/101460547 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f36900d2210 0x7f36900d2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.640+0000 7f36968d5640 1 --2- 172.21.15.87:0/101460547 >> v2:172.21.15.149:3300/0 conn(0x7f369015d870 0x7f36900ccf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.641 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.640+0000 7f36968d5640 1 -- 172.21.15.87:0/101460547 >> 172.21.15.87:0/101460547 conn(0x7f3690069120 msgr2=0x7f3690069560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:15.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.640+0000 7f36968d5640 1 -- 172.21.15.87:0/101460547 shutdown_connections 2024-09-17T03:16:15.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.640+0000 7f36968d5640 1 -- 172.21.15.87:0/101460547 wait complete. 2024-09-17T03:16:15.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.642+0000 7f36968d5640 1 Processor -- start 2024-09-17T03:16:15.642 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.642+0000 7f36968d5640 1 -- start start 2024-09-17T03:16:15.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.644+0000 7f36968d5640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f36900d2210 0x7f3690152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:15.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.644+0000 7f36968d5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f369015d870 0x7f3690154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:15.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.644+0000 7f36968d5640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3690160170 0x7f36901f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:15.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.644+0000 7f36968d5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3690152fc0 con 0x7f369015d870 2024-09-17T03:16:15.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.644+0000 7f36968d5640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f3690152e40 con 0x7f3690160170 2024-09-17T03:16:15.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.644+0000 7f36968d5640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f3690153140 con 0x7f36900d2210 2024-09-17T03:16:15.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.644+0000 7f368f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f369015d870 0x7f3690154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:15.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.644+0000 7f368f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f369015d870 0x7f3690154af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48762/0 (socket says 172.21.15.87:48762) 2024-09-17T03:16:15.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.644+0000 7f368f7fe640 1 -- 172.21.15.87:0/3806172920 learned_addr learned my addr 172.21.15.87:0/3806172920 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:15.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.644+0000 7f368ffff640 1 --2- 172.21.15.87:0/3806172920 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f36900d2210 0x7f3690152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:15.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.644+0000 7f3694e72640 1 --2- 172.21.15.87:0/3806172920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3690160170 0x7f36901f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:15.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.644+0000 7f368f7fe640 1 -- 172.21.15.87:0/3806172920 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f36900d2210 msgr2=0x7f3690152240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:15.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.644+0000 7f368f7fe640 1 --2- 172.21.15.87:0/3806172920 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f36900d2210 0x7f3690152240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.644+0000 7f368f7fe640 1 -- 172.21.15.87:0/3806172920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3690160170 msgr2=0x7f36901f73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:15.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.644+0000 7f368f7fe640 1 --2- 172.21.15.87:0/3806172920 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f3690160170 0x7f36901f73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.644+0000 7f368f7fe640 1 -- 172.21.15.87:0/3806172920 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36901f78e0 con 0x7f369015d870 2024-09-17T03:16:15.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.644+0000 7f368f7fe640 1 --2- 172.21.15.87:0/3806172920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f369015d870 0x7f3690154af0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f368400e7f0 tx=0x7f368400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:15.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.644+0000 7f368cff9640 1 -- 172.21.15.87:0/3806172920 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f368400eea0 con 0x7f369015d870 2024-09-17T03:16:15.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.644+0000 7f368cff9640 1 -- 172.21.15.87:0/3806172920 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f368400ccc0 con 0x7f369015d870 2024-09-17T03:16:15.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.644+0000 7f36968d5640 1 -- 172.21.15.87:0/3806172920 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36901f3690 con 0x7f369015d870 2024-09-17T03:16:15.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.644+0000 7f368cff9640 1 -- 172.21.15.87:0/3806172920 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3684005280 con 0x7f369015d870 2024-09-17T03:16:15.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.645+0000 7f36968d5640 1 -- 172.21.15.87:0/3806172920 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36900ca4e0 con 0x7f369015d870 2024-09-17T03:16:15.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.645+0000 7f36968d5640 1 -- 172.21.15.87:0/3806172920 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f36901f34b0 con 0x7f369015d870 2024-09-17T03:16:15.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.646+0000 7f368cff9640 1 -- 172.21.15.87:0/3806172920 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3684005420 con 0x7f369015d870 2024-09-17T03:16:15.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.647+0000 7f368cff9640 1 --2- 172.21.15.87:0/3806172920 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f367c078b90 0x7f367c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:15.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.647+0000 7f368cff9640 1 -- 172.21.15.87:0/3806172920 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f368409b250 con 0x7f369015d870 2024-09-17T03:16:15.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.647+0000 7f368cff9640 1 --2- 172.21.15.87:0/3806172920 >> [v2:172.21.15.149:6810/4289519146,v1:172.21.15.149:6811/4289519146] conn(0x7f367c07eaa0 0x7f367c080ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:15.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.647+0000 7f368cff9640 1 -- 172.21.15.87:0/3806172920 --> [v2:172.21.15.149:6810/4289519146,v1:172.21.15.149:6811/4289519146] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3684019e70 con 0x7f367c07eaa0 2024-09-17T03:16:15.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.647+0000 7f368cff9640 1 -- 172.21.15.87:0/3806172920 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f36840a2050 con 0x7f369015d870 2024-09-17T03:16:15.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.647+0000 7f368ffff640 1 --2- 172.21.15.87:0/3806172920 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f367c078b90 0x7f367c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:15.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.647+0000 7f3694e72640 1 --2- 172.21.15.87:0/3806172920 >> [v2:172.21.15.149:6810/4289519146,v1:172.21.15.149:6811/4289519146] conn(0x7f367c07eaa0 0x7f367c080ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:15.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.647+0000 7f368ffff640 1 --2- 172.21.15.87:0/3806172920 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f367c078b90 0x7f367c07b0a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3680009b30 tx=0x7f3680005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:15.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.647+0000 7f3694e72640 1 --2- 172.21.15.87:0/3806172920 >> [v2:172.21.15.149:6810/4289519146,v1:172.21.15.149:6811/4289519146] conn(0x7f367c07eaa0 0x7f367c080ef0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:15.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.648+0000 7f368cff9640 1 -- 172.21.15.87:0/3806172920 <== osd.6 v2:172.21.15.149:6810/4289519146 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3684019e70 con 0x7f367c07eaa0 2024-09-17T03:16:15.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.660+0000 7f36968d5640 1 -- 172.21.15.87:0/3806172920 --> [v2:172.21.15.149:6810/4289519146,v1:172.21.15.149:6811/4289519146] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3690156eb0 con 0x7f367c07eaa0 2024-09-17T03:16:15.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.661+0000 7f368cff9640 1 -- 172.21.15.87:0/3806172920 <== osd.6 v2:172.21.15.149:6810/4289519146 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3690156eb0 con 0x7f367c07eaa0 2024-09-17T03:16:15.661 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-17T03:16:15.662 INFO:teuthology.orchestra.run.smithi087.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T03:16:15.662 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-17T03:16:15.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.661+0000 7f36968d5640 1 -- 172.21.15.87:0/3806172920 >> [v2:172.21.15.149:6810/4289519146,v1:172.21.15.149:6811/4289519146] conn(0x7f367c07eaa0 msgr2=0x7f367c080ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:15.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.662+0000 7f36968d5640 1 --2- 172.21.15.87:0/3806172920 >> [v2:172.21.15.149:6810/4289519146,v1:172.21.15.149:6811/4289519146] conn(0x7f367c07eaa0 0x7f367c080ef0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.662+0000 7f36968d5640 1 -- 172.21.15.87:0/3806172920 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f367c078b90 msgr2=0x7f367c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:15.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.662+0000 7f36968d5640 1 --2- 172.21.15.87:0/3806172920 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f367c078b90 0x7f367c07b0a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3680009b30 tx=0x7f3680005f90 comp rx=0 tx=0).stop 2024-09-17T03:16:15.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.662+0000 7f36968d5640 1 -- 172.21.15.87:0/3806172920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f369015d870 msgr2=0x7f3690154af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:15.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.662+0000 7f36968d5640 1 --2- 172.21.15.87:0/3806172920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f369015d870 0x7f3690154af0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f368400e7f0 tx=0x7f368400ecb0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.662+0000 7f3694e72640 1 -- 172.21.15.87:0/3806172920 reap_dead start 2024-09-17T03:16:15.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.662+0000 7f36968d5640 1 -- 172.21.15.87:0/3806172920 shutdown_connections 2024-09-17T03:16:15.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.662+0000 7f36968d5640 1 -- 172.21.15.87:0/3806172920 >> 172.21.15.87:0/3806172920 conn(0x7f3690069120 msgr2=0x7f369015a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:15.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.662+0000 7f36968d5640 1 -- 172.21.15.87:0/3806172920 shutdown_connections 2024-09-17T03:16:15.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.662+0000 7f36968d5640 1 -- 172.21.15.87:0/3806172920 wait complete. 2024-09-17T03:16:15.672 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-17T03:16:15.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.787+0000 7f22df7cc640 1 -- 172.21.15.87:0/1815798590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22c4004800 msgr2=0x7f22c4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:15.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.787+0000 7f22df7cc640 1 --2- 172.21.15.87:0/1815798590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22c4004800 0x7f22c4004c10 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f22c8009b50 tx=0x7f22c4005830 comp rx=0 tx=0).stop 2024-09-17T03:16:15.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.787+0000 7f22ddd69640 1 -- 172.21.15.87:0/1815798590 reap_dead start 2024-09-17T03:16:15.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.787+0000 7f22df7cc640 1 -- 172.21.15.87:0/1815798590 shutdown_connections 2024-09-17T03:16:15.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.787+0000 7f22df7cc640 1 -- 172.21.15.87:0/1815798590 >> 172.21.15.87:0/1815798590 conn(0x7f22d808dae0 msgr2=0x7f22d809d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:15.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.787+0000 7f22df7cc640 1 -- 172.21.15.87:0/1815798590 shutdown_connections 2024-09-17T03:16:15.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.787+0000 7f22df7cc640 1 -- 172.21.15.87:0/1815798590 wait complete. 2024-09-17T03:16:15.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.789+0000 7f22df7cc640 1 Processor -- start 2024-09-17T03:16:15.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.789+0000 7f22df7cc640 1 -- start start 2024-09-17T03:16:15.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.790+0000 7f22df7cc640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22c4004800 0x7f22d80cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:15.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.790+0000 7f22df7cc640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22d814d600 0x7f22d814da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:15.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.790+0000 7f22df7cc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22d814df50 0x7f22d80b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:15.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.790+0000 7f22df7cc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f22d80d33f0 con 0x7f22d814df50 2024-09-17T03:16:15.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.790+0000 7f22df7cc640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f22d80d3270 con 0x7f22d814d600 2024-09-17T03:16:15.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.790+0000 7f22df7cc640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f22d80cf900 con 0x7f22c4004800 2024-09-17T03:16:15.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.790+0000 7f22ddd69640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22d814df50 0x7f22d80b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:15.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.790+0000 7f22ddd69640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22d814df50 0x7f22d80b2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48780/0 (socket says 172.21.15.87:48780) 2024-09-17T03:16:15.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.790+0000 7f22ddd69640 1 -- 172.21.15.87:0/2323972064 learned_addr learned my addr 172.21.15.87:0/2323972064 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:15.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.790+0000 7f22dd568640 1 --2- 172.21.15.87:0/2323972064 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22c4004800 0x7f22d80cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:15.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.790+0000 7f22dcd67640 1 --2- 172.21.15.87:0/2323972064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22d814d600 0x7f22d814da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:15.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.790+0000 7f22ddd69640 1 -- 172.21.15.87:0/2323972064 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22c4004800 msgr2=0x7f22d80cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:15.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.790+0000 7f22ddd69640 1 --2- 172.21.15.87:0/2323972064 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22c4004800 0x7f22d80cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.790+0000 7f22ddd69640 1 -- 172.21.15.87:0/2323972064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22d814d600 msgr2=0x7f22d814da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:15.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.790+0000 7f22ddd69640 1 --2- 172.21.15.87:0/2323972064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22d814d600 0x7f22d814da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.790+0000 7f22ddd69640 1 -- 172.21.15.87:0/2323972064 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d815a8b0 con 0x7f22d814df50 2024-09-17T03:16:15.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.791+0000 7f22ddd69640 1 --2- 172.21.15.87:0/2323972064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22d814df50 0x7f22d80b2050 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f22d80cb4d0 tx=0x7f22d0011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:15.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.791+0000 7f22cdffb640 1 -- 172.21.15.87:0/2323972064 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d0004b20 con 0x7f22d814df50 2024-09-17T03:16:15.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.791+0000 7f22cdffb640 1 -- 172.21.15.87:0/2323972064 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22d0004cc0 con 0x7f22d814df50 2024-09-17T03:16:15.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.791+0000 7f22cdffb640 1 -- 172.21.15.87:0/2323972064 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d0003bd0 con 0x7f22d814df50 2024-09-17T03:16:15.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.791+0000 7f22df7cc640 1 -- 172.21.15.87:0/2323972064 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d815a160 con 0x7f22d814df50 2024-09-17T03:16:15.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.791+0000 7f22df7cc640 1 -- 172.21.15.87:0/2323972064 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d815a5d0 con 0x7f22d814df50 2024-09-17T03:16:15.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.792+0000 7f22df7cc640 1 -- 172.21.15.87:0/2323972064 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22d815a330 con 0x7f22d814df50 2024-09-17T03:16:15.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.792+0000 7f22cdffb640 1 -- 172.21.15.87:0/2323972064 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f22d0004030 con 0x7f22d814df50 2024-09-17T03:16:15.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.793+0000 7f22cdffb640 1 --2- 172.21.15.87:0/2323972064 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f22bc078b90 0x7f22bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:15.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.793+0000 7f22cdffb640 1 -- 172.21.15.87:0/2323972064 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f22d009b580 con 0x7f22d814df50 2024-09-17T03:16:15.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.793+0000 7f22dd568640 1 --2- 172.21.15.87:0/2323972064 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f22bc078b90 0x7f22bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:15.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.794+0000 7f22dd568640 1 --2- 172.21.15.87:0/2323972064 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f22bc078b90 0x7f22bc07b0a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f22d4006c50 tx=0x7f22d40067a0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:15.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.798+0000 7f22cdffb640 1 -- 172.21.15.87:0/2323972064 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f22d815a330 con 0x7f22d814df50 2024-09-17T03:16:15.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.971+0000 7f22df7cc640 1 -- 172.21.15.87:0/2323972064 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f22d80cf3c0 con 0x7f22bc078b90 2024-09-17T03:16:15.971 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.971+0000 7f22cdffb640 1 -- 172.21.15.87:0/2323972064 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f22d80cf3c0 con 0x7f22bc078b90 2024-09-17T03:16:15.971 INFO:teuthology.orchestra.run.smithi087.stderr:instructed osd(s) 6 to deep-scrub 2024-09-17T03:16:15.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.974+0000 7f22df7cc640 1 -- 172.21.15.87:0/2323972064 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f22bc078b90 msgr2=0x7f22bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:15.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.974+0000 7f22df7cc640 1 --2- 172.21.15.87:0/2323972064 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f22bc078b90 0x7f22bc07b0a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f22d4006c50 tx=0x7f22d40067a0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.974+0000 7f22df7cc640 1 -- 172.21.15.87:0/2323972064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22d814df50 msgr2=0x7f22d80b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:15.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.974+0000 7f22df7cc640 1 --2- 172.21.15.87:0/2323972064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22d814df50 0x7f22d80b2050 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f22d80cb4d0 tx=0x7f22d0011f80 comp rx=0 tx=0).stop 2024-09-17T03:16:15.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.974+0000 7f22df7cc640 1 -- 172.21.15.87:0/2323972064 shutdown_connections 2024-09-17T03:16:15.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.974+0000 7f22df7cc640 1 --2- 172.21.15.87:0/2323972064 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f22bc078b90 0x7f22bc07b0a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.974+0000 7f22df7cc640 1 --2- 172.21.15.87:0/2323972064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22d814df50 0x7f22d80b2050 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.974+0000 7f22df7cc640 1 --2- 172.21.15.87:0/2323972064 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f22d814d600 0x7f22d814da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.974+0000 7f22df7cc640 1 --2- 172.21.15.87:0/2323972064 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f22c4004800 0x7f22d80cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:15.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.974+0000 7f22df7cc640 1 -- 172.21.15.87:0/2323972064 >> 172.21.15.87:0/2323972064 conn(0x7f22d808dae0 msgr2=0x7f22d815ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:15.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.974+0000 7f22df7cc640 1 -- 172.21.15.87:0/2323972064 shutdown_connections 2024-09-17T03:16:15.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:15.974+0000 7f22df7cc640 1 -- 172.21.15.87:0/2323972064 wait complete. 2024-09-17T03:16:15.985 INFO:tasks.ceph:Scrubbing osd.7 2024-09-17T03:16:15.985 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-17T03:16:16.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.102+0000 7f81a3c8b640 1 -- 172.21.15.87:0/156435611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f819c0d0420 msgr2=0x7f819c15f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:16.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.102+0000 7f81a3c8b640 1 --2- 172.21.15.87:0/156435611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f819c0d0420 0x7f819c15f050 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f819c073620 tx=0x7f819802a820 comp rx=0 tx=0).stop 2024-09-17T03:16:16.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.102+0000 7f81a3c8b640 1 -- 172.21.15.87:0/156435611 shutdown_connections 2024-09-17T03:16:16.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.102+0000 7f81a3c8b640 1 --2- 172.21.15.87:0/156435611 >> v2:172.21.15.149:3300/0 conn(0x7f819c0afee0 0x7f819c0ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:16.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.102+0000 7f81a3c8b640 1 --2- 172.21.15.87:0/156435611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f819c0d0420 0x7f819c15f050 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:16.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.102+0000 7f81a3c8b640 1 -- 172.21.15.87:0/156435611 >> 172.21.15.87:0/156435611 conn(0x7f819c0cd350 msgr2=0x7f819c0cd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:16.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.102+0000 7f81a3c8b640 1 -- 172.21.15.87:0/156435611 shutdown_connections 2024-09-17T03:16:16.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.102+0000 7f81a3c8b640 1 -- 172.21.15.87:0/156435611 wait complete. 2024-09-17T03:16:16.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.104+0000 7f81a3c8b640 1 Processor -- start 2024-09-17T03:16:16.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.104+0000 7f81a3c8b640 1 -- start start 2024-09-17T03:16:16.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.105+0000 7f81a3c8b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f819c0afee0 0x7f819c1f74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:16.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.105+0000 7f81a3c8b640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f819c0d0420 0x7f819c1f3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:16.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.105+0000 7f81a3c8b640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f819c155cb0 0x7f819c144280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:16.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.105+0000 7f81a3c8b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f819c0cf6c0 con 0x7f819c0afee0 2024-09-17T03:16:16.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.105+0000 7f81a3c8b640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f819c0cf540 con 0x7f819c0d0420 2024-09-17T03:16:16.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.105+0000 7f81a3c8b640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f819c0cf840 con 0x7f819c155cb0 2024-09-17T03:16:16.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.105+0000 7f81a1a27640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f819c0afee0 0x7f819c1f74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:16.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.105+0000 7f81a1a27640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f819c0afee0 0x7f819c1f74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48804/0 (socket says 172.21.15.87:48804) 2024-09-17T03:16:16.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.105+0000 7f81a1a27640 1 -- 172.21.15.87:0/1591760501 learned_addr learned my addr 172.21.15.87:0/1591760501 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:16.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.105+0000 7f81a2228640 1 --2- 172.21.15.87:0/1591760501 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f819c155cb0 0x7f819c144280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:16.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.105+0000 7f81a1226640 1 --2- 172.21.15.87:0/1591760501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f819c0d0420 0x7f819c1f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:16.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.106+0000 7f81a1a27640 1 -- 172.21.15.87:0/1591760501 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f819c155cb0 msgr2=0x7f819c144280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:16.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.106+0000 7f81a1a27640 1 --2- 172.21.15.87:0/1591760501 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f819c155cb0 0x7f819c144280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:16.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.106+0000 7f81a1a27640 1 -- 172.21.15.87:0/1591760501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f819c0d0420 msgr2=0x7f819c1f3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:16.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.106+0000 7f81a1a27640 1 --2- 172.21.15.87:0/1591760501 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f819c0d0420 0x7f819c1f3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:16.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.106+0000 7f81a1a27640 1 -- 172.21.15.87:0/1591760501 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819c1f7a20 con 0x7f819c0afee0 2024-09-17T03:16:16.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.106+0000 7f81a1a27640 1 --2- 172.21.15.87:0/1591760501 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f819c0afee0 0x7f819c1f74e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8198035040 tx=0x7f8198032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:16.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.106+0000 7f81927fc640 1 -- 172.21.15.87:0/1591760501 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198037040 con 0x7f819c0afee0 2024-09-17T03:16:16.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.106+0000 7f81927fc640 1 -- 172.21.15.87:0/1591760501 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8198042070 con 0x7f819c0afee0 2024-09-17T03:16:16.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.106+0000 7f81927fc640 1 -- 172.21.15.87:0/1591760501 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8198003e60 con 0x7f819c0afee0 2024-09-17T03:16:16.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.106+0000 7f81a3c8b640 1 -- 172.21.15.87:0/1591760501 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f819c1f3fe0 con 0x7f819c0afee0 2024-09-17T03:16:16.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.106+0000 7f81a3c8b640 1 -- 172.21.15.87:0/1591760501 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f819c1f6ce0 con 0x7f819c0afee0 2024-09-17T03:16:16.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.107+0000 7f81a3c8b640 1 -- 172.21.15.87:0/1591760501 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f819c1f70c0 con 0x7f819c0afee0 2024-09-17T03:16:16.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.108+0000 7f81927fc640 1 -- 172.21.15.87:0/1591760501 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8198004280 con 0x7f819c0afee0 2024-09-17T03:16:16.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.108+0000 7f81927fc640 1 --2- 172.21.15.87:0/1591760501 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8180078b90 0x7f818007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:16.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.108+0000 7f81927fc640 1 -- 172.21.15.87:0/1591760501 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f81980bc200 con 0x7f819c0afee0 2024-09-17T03:16:16.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.108+0000 7f81a1226640 1 --2- 172.21.15.87:0/1591760501 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8180078b90 0x7f818007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:16.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.109+0000 7f81927fc640 1 --2- 172.21.15.87:0/1591760501 >> [v2:172.21.15.149:6818/2771010477,v1:172.21.15.149:6819/2771010477] conn(0x7f818007eaa0 0x7f8180080ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:16.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.109+0000 7f81927fc640 1 -- 172.21.15.87:0/1591760501 --> [v2:172.21.15.149:6818/2771010477,v1:172.21.15.149:6819/2771010477] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f81800815c0 con 0x7f818007eaa0 2024-09-17T03:16:16.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.109+0000 7f81927fc640 1 -- 172.21.15.87:0/1591760501 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f81980bc690 con 0x7f819c0afee0 2024-09-17T03:16:16.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.109+0000 7f81a2228640 1 --2- 172.21.15.87:0/1591760501 >> [v2:172.21.15.149:6818/2771010477,v1:172.21.15.149:6819/2771010477] conn(0x7f818007eaa0 0x7f8180080ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:16.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.109+0000 7f81a1226640 1 --2- 172.21.15.87:0/1591760501 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8180078b90 0x7f818007b0a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f819c1555a0 tx=0x7f818c009040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:16.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.109+0000 7f81a2228640 1 --2- 172.21.15.87:0/1591760501 >> [v2:172.21.15.149:6818/2771010477,v1:172.21.15.149:6819/2771010477] conn(0x7f818007eaa0 0x7f8180080ef0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:16.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.110+0000 7f81927fc640 1 -- 172.21.15.87:0/1591760501 <== osd.7 v2:172.21.15.149:6818/2771010477 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f81800815c0 con 0x7f818007eaa0 2024-09-17T03:16:16.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.122+0000 7f81a3c8b640 1 -- 172.21.15.87:0/1591760501 --> [v2:172.21.15.149:6818/2771010477,v1:172.21.15.149:6819/2771010477] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f819c0cfcf0 con 0x7f818007eaa0 2024-09-17T03:16:16.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.123+0000 7f81927fc640 1 -- 172.21.15.87:0/1591760501 <== osd.7 v2:172.21.15.149:6818/2771010477 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f819c0cfcf0 con 0x7f818007eaa0 2024-09-17T03:16:16.122 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-09-17T03:16:16.123 INFO:teuthology.orchestra.run.smithi087.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-17T03:16:16.123 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-09-17T03:16:16.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.123+0000 7f81a3c8b640 1 -- 172.21.15.87:0/1591760501 >> [v2:172.21.15.149:6818/2771010477,v1:172.21.15.149:6819/2771010477] conn(0x7f818007eaa0 msgr2=0x7f8180080ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:16.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.123+0000 7f81a3c8b640 1 --2- 172.21.15.87:0/1591760501 >> [v2:172.21.15.149:6818/2771010477,v1:172.21.15.149:6819/2771010477] conn(0x7f818007eaa0 0x7f8180080ef0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:16.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.123+0000 7f81a3c8b640 1 -- 172.21.15.87:0/1591760501 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8180078b90 msgr2=0x7f818007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:16.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.123+0000 7f81a3c8b640 1 --2- 172.21.15.87:0/1591760501 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f8180078b90 0x7f818007b0a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f819c1555a0 tx=0x7f818c009040 comp rx=0 tx=0).stop 2024-09-17T03:16:16.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.123+0000 7f81a3c8b640 1 -- 172.21.15.87:0/1591760501 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f819c0afee0 msgr2=0x7f819c1f74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:16.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.123+0000 7f81a3c8b640 1 --2- 172.21.15.87:0/1591760501 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f819c0afee0 0x7f819c1f74e0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f8198035040 tx=0x7f8198032d10 comp rx=0 tx=0).stop 2024-09-17T03:16:16.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.123+0000 7f81a2228640 1 -- 172.21.15.87:0/1591760501 reap_dead start 2024-09-17T03:16:16.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.123+0000 7f81a3c8b640 1 -- 172.21.15.87:0/1591760501 shutdown_connections 2024-09-17T03:16:16.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.123+0000 7f81a3c8b640 1 -- 172.21.15.87:0/1591760501 >> 172.21.15.87:0/1591760501 conn(0x7f819c0cd350 msgr2=0x7f819c15dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:16.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.123+0000 7f81a3c8b640 1 -- 172.21.15.87:0/1591760501 shutdown_connections 2024-09-17T03:16:16.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.123+0000 7f81a3c8b640 1 -- 172.21.15.87:0/1591760501 wait complete. 2024-09-17T03:16:16.134 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-17T03:16:16.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.247+0000 7f714ed6e640 1 -- 172.21.15.87:0/2550608211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71480d1080 msgr2=0x7f71480ccf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:16.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.247+0000 7f714ed6e640 1 --2- 172.21.15.87:0/2550608211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71480d1080 0x7f71480ccf80 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f71480d1d80 tx=0x7f714002aa10 comp rx=0 tx=0).stop 2024-09-17T03:16:16.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.247+0000 7f714ed6e640 1 -- 172.21.15.87:0/2550608211 shutdown_connections 2024-09-17T03:16:16.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.247+0000 7f714ed6e640 1 --2- 172.21.15.87:0/2550608211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71480d1080 0x7f71480ccf80 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:16.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.247+0000 7f714ed6e640 1 --2- 172.21.15.87:0/2550608211 >> v2:172.21.15.149:3300/0 conn(0x7f71480cf340 0x7f71480d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:16.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.247+0000 7f714ed6e640 1 --2- 172.21.15.87:0/2550608211 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f71480b76d0 0x7f71480cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:16.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.247+0000 7f714ed6e640 1 -- 172.21.15.87:0/2550608211 >> 172.21.15.87:0/2550608211 conn(0x7f71480d2230 msgr2=0x7f71480d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:16.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.247+0000 7f714ed6e640 1 -- 172.21.15.87:0/2550608211 shutdown_connections 2024-09-17T03:16:16.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.247+0000 7f714ed6e640 1 -- 172.21.15.87:0/2550608211 wait complete. 2024-09-17T03:16:16.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.249+0000 7f714ed6e640 1 Processor -- start 2024-09-17T03:16:16.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.249+0000 7f714ed6e640 1 -- start start 2024-09-17T03:16:16.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.251+0000 7f714ed6e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71480b76d0 0x7f714809f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:16.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.251+0000 7f714ed6e640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71480cf340 0x7f714809cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:16.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.251+0000 7f714ed6e640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f71480d1080 0x7f714809f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:16.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.251+0000 7f714ed6e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f71480cd4c0 con 0x7f71480b76d0 2024-09-17T03:16:16.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.251+0000 7f714ed6e640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f71480b30d0 con 0x7f71480cf340 2024-09-17T03:16:16.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.251+0000 7f714ed6e640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f71480cd640 con 0x7f71480d1080 2024-09-17T03:16:16.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.251+0000 7f714cb0a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71480b76d0 0x7f714809f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:16.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.251+0000 7f714cb0a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71480b76d0 0x7f714809f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48828/0 (socket says 172.21.15.87:48828) 2024-09-17T03:16:16.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.251+0000 7f714cb0a640 1 -- 172.21.15.87:0/3290632928 learned_addr learned my addr 172.21.15.87:0/3290632928 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:16.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.251+0000 7f714d30b640 1 --2- 172.21.15.87:0/3290632928 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f71480d1080 0x7f714809f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:16.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.251+0000 7f7147fff640 1 --2- 172.21.15.87:0/3290632928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71480cf340 0x7f714809cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:16.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.251+0000 7f714cb0a640 1 -- 172.21.15.87:0/3290632928 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f71480d1080 msgr2=0x7f714809f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:16.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.251+0000 7f714cb0a640 1 --2- 172.21.15.87:0/3290632928 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f71480d1080 0x7f714809f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:16.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.251+0000 7f714cb0a640 1 -- 172.21.15.87:0/3290632928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71480cf340 msgr2=0x7f714809cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:16.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.251+0000 7f714cb0a640 1 --2- 172.21.15.87:0/3290632928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71480cf340 0x7f714809cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:16.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.251+0000 7f714cb0a640 1 -- 172.21.15.87:0/3290632928 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714814d780 con 0x7f71480b76d0 2024-09-17T03:16:16.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.251+0000 7f714d30b640 1 --2- 172.21.15.87:0/3290632928 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f71480d1080 0x7f714809f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:16:16.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.251+0000 7f7147fff640 1 --2- 172.21.15.87:0/3290632928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71480cf340 0x7f714809cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-17T03:16:16.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.251+0000 7f714cb0a640 1 --2- 172.21.15.87:0/3290632928 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71480b76d0 0x7f714809f220 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f713c00c940 tx=0x7f713c00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:16.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.251+0000 7f71457fa640 1 -- 172.21.15.87:0/3290632928 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c007bb0 con 0x7f71480b76d0 2024-09-17T03:16:16.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.251+0000 7f714ed6e640 1 -- 172.21.15.87:0/3290632928 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714814da60 con 0x7f71480b76d0 2024-09-17T03:16:16.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.251+0000 7f71457fa640 1 -- 172.21.15.87:0/3290632928 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f713c007e40 con 0x7f71480b76d0 2024-09-17T03:16:16.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.251+0000 7f71457fa640 1 -- 172.21.15.87:0/3290632928 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713c00f670 con 0x7f71480b76d0 2024-09-17T03:16:16.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.251+0000 7f714ed6e640 1 -- 172.21.15.87:0/3290632928 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f714814e020 con 0x7f71480b76d0 2024-09-17T03:16:16.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.252+0000 7f714ed6e640 1 -- 172.21.15.87:0/3290632928 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71480719a0 con 0x7f71480b76d0 2024-09-17T03:16:16.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.253+0000 7f71457fa640 1 -- 172.21.15.87:0/3290632928 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f713c00f810 con 0x7f71480b76d0 2024-09-17T03:16:16.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.254+0000 7f71457fa640 1 --2- 172.21.15.87:0/3290632928 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f712c078b90 0x7f712c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:16.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.254+0000 7f71457fa640 1 -- 172.21.15.87:0/3290632928 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f713c09b270 con 0x7f71480b76d0 2024-09-17T03:16:16.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.254+0000 7f7147fff640 1 --2- 172.21.15.87:0/3290632928 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f712c078b90 0x7f712c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:16.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.254+0000 7f7147fff640 1 --2- 172.21.15.87:0/3290632928 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f712c078b90 0x7f712c07b0a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f714809cad0 tx=0x7f7134005cf0 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:16.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.258+0000 7f71457fa640 1 -- 172.21.15.87:0/3290632928 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f713c014030 con 0x7f71480b76d0 2024-09-17T03:16:16.433 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.434+0000 7f714ed6e640 1 -- 172.21.15.87:0/3290632928 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f71480d29c0 con 0x7f712c078b90 2024-09-17T03:16:16.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.434+0000 7f71457fa640 1 -- 172.21.15.87:0/3290632928 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f71480d29c0 con 0x7f712c078b90 2024-09-17T03:16:16.434 INFO:teuthology.orchestra.run.smithi087.stderr:instructed osd(s) 7 to deep-scrub 2024-09-17T03:16:16.436 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.436+0000 7f714ed6e640 1 -- 172.21.15.87:0/3290632928 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f712c078b90 msgr2=0x7f712c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:16.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.436+0000 7f714ed6e640 1 --2- 172.21.15.87:0/3290632928 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f712c078b90 0x7f712c07b0a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f714809cad0 tx=0x7f7134005cf0 comp rx=0 tx=0).stop 2024-09-17T03:16:16.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.436+0000 7f714ed6e640 1 -- 172.21.15.87:0/3290632928 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71480b76d0 msgr2=0x7f714809f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:16.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.436+0000 7f714ed6e640 1 --2- 172.21.15.87:0/3290632928 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71480b76d0 0x7f714809f220 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f713c00c940 tx=0x7f713c00ce00 comp rx=0 tx=0).stop 2024-09-17T03:16:16.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.437+0000 7f714ed6e640 1 -- 172.21.15.87:0/3290632928 shutdown_connections 2024-09-17T03:16:16.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.437+0000 7f714ed6e640 1 --2- 172.21.15.87:0/3290632928 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f712c078b90 0x7f712c07b0a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:16.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.437+0000 7f714ed6e640 1 --2- 172.21.15.87:0/3290632928 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f71480d1080 0x7f714809f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:16.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.437+0000 7f714ed6e640 1 --2- 172.21.15.87:0/3290632928 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f71480cf340 0x7f714809cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:16.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.437+0000 7f714ed6e640 1 --2- 172.21.15.87:0/3290632928 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71480b76d0 0x7f714809f220 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:16.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.437+0000 7f714ed6e640 1 -- 172.21.15.87:0/3290632928 >> 172.21.15.87:0/3290632928 conn(0x7f71480d2230 msgr2=0x7f71480b53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:16.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.437+0000 7f714ed6e640 1 -- 172.21.15.87:0/3290632928 shutdown_connections 2024-09-17T03:16:16.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.437+0000 7f714ed6e640 1 -- 172.21.15.87:0/3290632928 wait complete. 2024-09-17T03:16:16.448 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:16:16.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.560+0000 7fd095030640 1 -- 172.21.15.87:0/1092068010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd078004800 msgr2=0x7fd078004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:16.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.560+0000 7fd095030640 1 --2- 172.21.15.87:0/1092068010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd078004800 0x7fd078004c10 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd0840016c0 tx=0x7fd078005830 comp rx=0 tx=0).stop 2024-09-17T03:16:16.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.560+0000 7fd095030640 1 -- 172.21.15.87:0/1092068010 shutdown_connections 2024-09-17T03:16:16.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.560+0000 7fd095030640 1 --2- 172.21.15.87:0/1092068010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd078004800 0x7fd078004c10 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:16.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.560+0000 7fd095030640 1 -- 172.21.15.87:0/1092068010 >> 172.21.15.87:0/1092068010 conn(0x7fd0900d1180 msgr2=0x7fd0900d1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:16.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.560+0000 7fd095030640 1 -- 172.21.15.87:0/1092068010 shutdown_connections 2024-09-17T03:16:16.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.560+0000 7fd095030640 1 -- 172.21.15.87:0/1092068010 wait complete. 2024-09-17T03:16:16.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.562+0000 7fd095030640 1 Processor -- start 2024-09-17T03:16:16.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.562+0000 7fd095030640 1 -- start start 2024-09-17T03:16:16.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.564+0000 7fd095030640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd078004800 0x7fd090156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:16.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.564+0000 7fd095030640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0900d0a50 0x7fd090149fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:16.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.564+0000 7fd095030640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd090153070 0x7fd090153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:16.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.564+0000 7fd095030640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd0900b4110 con 0x7fd0900d0a50 2024-09-17T03:16:16.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.564+0000 7fd095030640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fd09015f840 con 0x7fd078004800 2024-09-17T03:16:16.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.564+0000 7fd095030640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fd0900b4290 con 0x7fd090153070 2024-09-17T03:16:16.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.564+0000 7fd08e59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0900d0a50 0x7fd090149fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:16.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.564+0000 7fd08e59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0900d0a50 0x7fd090149fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48848/0 (socket says 172.21.15.87:48848) 2024-09-17T03:16:16.565 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.564+0000 7fd08e59c640 1 -- 172.21.15.87:0/1196796435 learned_addr learned my addr 172.21.15.87:0/1196796435 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:16.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.564+0000 7fd08f59e640 1 --2- 172.21.15.87:0/1196796435 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd090153070 0x7fd090153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:16.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.564+0000 7fd08ed9d640 1 --2- 172.21.15.87:0/1196796435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd078004800 0x7fd090156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:16.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.564+0000 7fd08e59c640 1 -- 172.21.15.87:0/1196796435 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd090153070 msgr2=0x7fd090153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:16.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.564+0000 7fd08e59c640 1 --2- 172.21.15.87:0/1196796435 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd090153070 0x7fd090153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:16.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.564+0000 7fd08e59c640 1 -- 172.21.15.87:0/1196796435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd078004800 msgr2=0x7fd090156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:16.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.564+0000 7fd08e59c640 1 --2- 172.21.15.87:0/1196796435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd078004800 0x7fd090156fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:16.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.564+0000 7fd08e59c640 1 -- 172.21.15.87:0/1196796435 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0901539c0 con 0x7fd0900d0a50 2024-09-17T03:16:16.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.564+0000 7fd08e59c640 1 --2- 172.21.15.87:0/1196796435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0900d0a50 0x7fd090149fc0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd0840017e0 tx=0x7fd0840063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:16.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.564+0000 7fd0777fe640 1 -- 172.21.15.87:0/1196796435 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd084042070 con 0x7fd0900d0a50 2024-09-17T03:16:16.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.564+0000 7fd0777fe640 1 -- 172.21.15.87:0/1196796435 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd084039070 con 0x7fd0900d0a50 2024-09-17T03:16:16.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.564+0000 7fd095030640 1 -- 172.21.15.87:0/1196796435 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd09014a500 con 0x7fd0900d0a50 2024-09-17T03:16:16.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.564+0000 7fd0777fe640 1 -- 172.21.15.87:0/1196796435 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd084003d70 con 0x7fd0900d0a50 2024-09-17T03:16:16.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.565+0000 7fd095030640 1 -- 172.21.15.87:0/1196796435 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd090156200 con 0x7fd0900d0a50 2024-09-17T03:16:16.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.565+0000 7fd095030640 1 -- 172.21.15.87:0/1196796435 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0900b47f0 con 0x7fd0900d0a50 2024-09-17T03:16:16.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.566+0000 7fd0777fe640 1 -- 172.21.15.87:0/1196796435 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd08403b070 con 0x7fd0900d0a50 2024-09-17T03:16:16.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.567+0000 7fd0777fe640 1 --2- 172.21.15.87:0/1196796435 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd07c078b90 0x7fd07c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:16.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.567+0000 7fd0777fe640 1 -- 172.21.15.87:0/1196796435 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7fd0840b9f00 con 0x7fd0900d0a50 2024-09-17T03:16:16.568 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.567+0000 7fd08ed9d640 1 --2- 172.21.15.87:0/1196796435 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd07c078b90 0x7fd07c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:16.569 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.568+0000 7fd08ed9d640 1 --2- 172.21.15.87:0/1196796435 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd07c078b90 0x7fd07c07b0a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd0800015e0 tx=0x7fd080006040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:16.571 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.571+0000 7fd0777fe640 1 -- 172.21.15.87:0/1196796435 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fd09014a500 con 0x7fd0900d0a50 2024-09-17T03:16:16.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.742+0000 7fd095030640 1 -- 172.21.15.87:0/1196796435 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd090075dc0 con 0x7fd07c078b90 2024-09-17T03:16:16.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.743+0000 7fd0777fe640 1 -- 172.21.15.87:0/1196796435 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13460 (secure 0 0 0) 0x7fd090075dc0 con 0x7fd07c078b90 2024-09-17T03:16:16.743 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:16:16.743 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-09-17T03:16:16.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.745+0000 7fd095030640 1 -- 172.21.15.87:0/1196796435 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd07c078b90 msgr2=0x7fd07c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:16.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.745+0000 7fd095030640 1 --2- 172.21.15.87:0/1196796435 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd07c078b90 0x7fd07c07b0a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd0800015e0 tx=0x7fd080006040 comp rx=0 tx=0).stop 2024-09-17T03:16:16.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.745+0000 7fd095030640 1 -- 172.21.15.87:0/1196796435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0900d0a50 msgr2=0x7fd090149fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:16.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.745+0000 7fd095030640 1 --2- 172.21.15.87:0/1196796435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0900d0a50 0x7fd090149fc0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd0840017e0 tx=0x7fd0840063a0 comp rx=0 tx=0).stop 2024-09-17T03:16:16.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.745+0000 7fd095030640 1 -- 172.21.15.87:0/1196796435 shutdown_connections 2024-09-17T03:16:16.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.745+0000 7fd095030640 1 --2- 172.21.15.87:0/1196796435 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fd07c078b90 0x7fd07c07b0a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:16.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.745+0000 7fd095030640 1 --2- 172.21.15.87:0/1196796435 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fd090153070 0x7fd090153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:16.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.745+0000 7fd095030640 1 --2- 172.21.15.87:0/1196796435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0900d0a50 0x7fd090149fc0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:16.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.745+0000 7fd095030640 1 --2- 172.21.15.87:0/1196796435 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fd078004800 0x7fd090156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:16.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.745+0000 7fd095030640 1 -- 172.21.15.87:0/1196796435 >> 172.21.15.87:0/1196796435 conn(0x7fd0900d1180 msgr2=0x7fd09014be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:16.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.745+0000 7fd095030640 1 -- 172.21.15.87:0/1196796435 shutdown_connections 2024-09-17T03:16:16.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:16.745+0000 7fd095030640 1 -- 172.21.15.87:0/1196796435 wait complete. 2024-09-17T03:16:16.756 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2024-09-17T03:16:15.447412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":54,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":230856,"kb_used_data":6280,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754743864,"statfs":{"total":773094113280,"available":772857716736,"internally_reserved":0,"allocated":6430720,"data_stored":3342872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14792,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-65,"num_write_kb":-36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001554"},"pg_stats":[{"pgid":"1.0","version":"18'72","reported_seq":170,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T03:16:09.064637+0000","last_change":"2024-09-17T03:14:04.862167+0000","last_active":"2024-09-17T03:16:09.064637+0000","last_peered":"2024-09-17T03:16:09.064637+0000","last_clean":"2024-09-17T03:16:09.064637+0000","last_became_active":"2024-09-17T03:14:04.861731+0000","last_became_peered":"2024-09-17T03:14:04.861731+0000","last_unstale":"2024-09-17T03:16:09.064637+0000","last_undegraded":"2024-09-17T03:16:09.064637+0000","last_fullsized":"2024-09-17T03:16:09.064637+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-17T03:14:03.847721+0000","last_clean_scrub_stamp":"2024-09-17T03:14:03.847721+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:21:32.024357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509471,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27672,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":655360,"data_stored":270267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509471,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27676,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":655360,"data_stored":270267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509470,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1249280,"data_stored":860635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509471,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27672,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":655360,"data_stored":270267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509471,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27672,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":655360,"data_stored":270267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509471,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1249280,"data_stored":860635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509470,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27672,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":655360,"data_stored":270267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574878,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31832,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":655360,"data_stored":270267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:16:16.758 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-17T03:14:03.847721+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=3, tm_min=14, tm_sec=3, tm_wday=1, tm_yday=261, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=17, tm_hour=3, tm_min=16, tm_sec=11, tm_wday=1, tm_yday=261, tm_isdst=0) 2024-09-17T03:16:16.759 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-17T03:16:36.760 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-17T03:16:36.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.888+0000 7ff0e84b6640 1 -- 172.21.15.87:0/1608379596 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0cc003a30 msgr2=0x7ff0cc003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:36.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.888+0000 7ff0e84b6640 1 --2- 172.21.15.87:0/1608379596 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0cc003a30 0x7ff0cc003ea0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc00c2e0 tx=0x7ff0cc005830 comp rx=0 tx=0).stop 2024-09-17T03:16:36.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.888+0000 7ff0e6a53640 1 -- 172.21.15.87:0/1608379596 reap_dead start 2024-09-17T03:16:36.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.888+0000 7ff0e84b6640 1 -- 172.21.15.87:0/1608379596 shutdown_connections 2024-09-17T03:16:36.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.888+0000 7ff0e84b6640 1 -- 172.21.15.87:0/1608379596 >> 172.21.15.87:0/1608379596 conn(0x7ff0e00b20f0 msgr2=0x7ff0e00b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:36.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.888+0000 7ff0e84b6640 1 -- 172.21.15.87:0/1608379596 shutdown_connections 2024-09-17T03:16:36.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.888+0000 7ff0e84b6640 1 -- 172.21.15.87:0/1608379596 wait complete. 2024-09-17T03:16:36.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.890+0000 7ff0e84b6640 1 Processor -- start 2024-09-17T03:16:36.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.890+0000 7ff0e84b6640 1 -- start start 2024-09-17T03:16:36.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.891+0000 7ff0e84b6640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff0cc003a30 0x7ff0e01f18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:36.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.891+0000 7ff0e84b6640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0e00b5ad0 0x7ff0e0142a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:36.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.891+0000 7ff0e84b6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0e0142fc0 0x7ff0e01f5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:36.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.891+0000 7ff0e84b6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e00cf2d0 con 0x7ff0e0142fc0 2024-09-17T03:16:36.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.891+0000 7ff0e84b6640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e00cf150 con 0x7ff0e00b5ad0 2024-09-17T03:16:36.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.891+0000 7ff0e84b6640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7ff0e00cb980 con 0x7ff0cc003a30 2024-09-17T03:16:36.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.891+0000 7ff0e6a53640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0e0142fc0 0x7ff0e01f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:36.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.892+0000 7ff0e6a53640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0e0142fc0 0x7ff0e01f5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59376/0 (socket says 172.21.15.87:59376) 2024-09-17T03:16:36.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.892+0000 7ff0e6a53640 1 -- 172.21.15.87:0/1877797060 learned_addr learned my addr 172.21.15.87:0/1877797060 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:36.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.892+0000 7ff0e6252640 1 --2- 172.21.15.87:0/1877797060 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff0cc003a30 0x7ff0e01f18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:36.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.892+0000 7ff0e5a51640 1 --2- 172.21.15.87:0/1877797060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0e00b5ad0 0x7ff0e0142a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:36.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.892+0000 7ff0e6a53640 1 -- 172.21.15.87:0/1877797060 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff0cc003a30 msgr2=0x7ff0e01f18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:36.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.892+0000 7ff0e6a53640 1 --2- 172.21.15.87:0/1877797060 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff0cc003a30 0x7ff0e01f18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:36.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.892+0000 7ff0e6a53640 1 -- 172.21.15.87:0/1877797060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0e00b5ad0 msgr2=0x7ff0e0142a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:36.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.892+0000 7ff0e6a53640 1 --2- 172.21.15.87:0/1877797060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0e00b5ad0 0x7ff0e0142a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:36.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.892+0000 7ff0e6a53640 1 -- 172.21.15.87:0/1877797060 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e01f1de0 con 0x7ff0e0142fc0 2024-09-17T03:16:36.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.892+0000 7ff0e6a53640 1 --2- 172.21.15.87:0/1877797060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0e0142fc0 0x7ff0e01f5590 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0160160 tx=0x7ff0d80019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:36.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.892+0000 7ff0d6ffd640 1 -- 172.21.15.87:0/1877797060 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8017070 con 0x7ff0e0142fc0 2024-09-17T03:16:36.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.892+0000 7ff0d6ffd640 1 -- 172.21.15.87:0/1877797060 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0d80108c0 con 0x7ff0e0142fc0 2024-09-17T03:16:36.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.892+0000 7ff0e84b6640 1 -- 172.21.15.87:0/1877797060 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0e00ca120 con 0x7ff0e0142fc0 2024-09-17T03:16:36.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.892+0000 7ff0d6ffd640 1 -- 172.21.15.87:0/1877797060 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8010d10 con 0x7ff0e0142fc0 2024-09-17T03:16:36.894 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.892+0000 7ff0e84b6640 1 -- 172.21.15.87:0/1877797060 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0e00b52d0 con 0x7ff0e0142fc0 2024-09-17T03:16:36.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.893+0000 7ff0e84b6640 1 -- 172.21.15.87:0/1877797060 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0e00c9e90 con 0x7ff0e0142fc0 2024-09-17T03:16:36.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.894+0000 7ff0d6ffd640 1 -- 172.21.15.87:0/1877797060 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff0d801a070 con 0x7ff0e0142fc0 2024-09-17T03:16:36.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.895+0000 7ff0d6ffd640 1 --2- 172.21.15.87:0/1877797060 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff0c4078bc0 0x7ff0c407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:36.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.895+0000 7ff0d6ffd640 1 -- 172.21.15.87:0/1877797060 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7ff0d809a650 con 0x7ff0e0142fc0 2024-09-17T03:16:36.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.895+0000 7ff0e6252640 1 --2- 172.21.15.87:0/1877797060 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff0c4078bc0 0x7ff0c407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:36.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.896+0000 7ff0e6252640 1 --2- 172.21.15.87:0/1877797060 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff0c4078bc0 0x7ff0c407b0d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc033000 tx=0x7ff0dc038040 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:36.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:36.899+0000 7ff0d6ffd640 1 -- 172.21.15.87:0/1877797060 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7ff0d80627d0 con 0x7ff0e0142fc0 2024-09-17T03:16:37.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.070+0000 7ff0e84b6640 1 -- 172.21.15.87:0/1877797060 --> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff0e015f0e0 con 0x7ff0c4078bc0 2024-09-17T03:16:37.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.071+0000 7ff0d6ffd640 1 -- 172.21.15.87:0/1877797060 <== mgr.4117 v2:172.21.15.149:6800/973360146 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13344 (secure 0 0 0) 0x7ff0e015f0e0 con 0x7ff0c4078bc0 2024-09-17T03:16:37.070 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:16:37.070 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-09-17T03:16:37.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.073+0000 7ff0e84b6640 1 -- 172.21.15.87:0/1877797060 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff0c4078bc0 msgr2=0x7ff0c407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:37.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.073+0000 7ff0e84b6640 1 --2- 172.21.15.87:0/1877797060 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff0c4078bc0 0x7ff0c407b0d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc033000 tx=0x7ff0dc038040 comp rx=0 tx=0).stop 2024-09-17T03:16:37.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.073+0000 7ff0e84b6640 1 -- 172.21.15.87:0/1877797060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0e0142fc0 msgr2=0x7ff0e01f5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:37.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.073+0000 7ff0e84b6640 1 --2- 172.21.15.87:0/1877797060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0e0142fc0 0x7ff0e01f5590 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0160160 tx=0x7ff0d80019b0 comp rx=0 tx=0).stop 2024-09-17T03:16:37.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.073+0000 7ff0e84b6640 1 -- 172.21.15.87:0/1877797060 shutdown_connections 2024-09-17T03:16:37.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.073+0000 7ff0e84b6640 1 --2- 172.21.15.87:0/1877797060 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7ff0c4078bc0 0x7ff0c407b0d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:37.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.073+0000 7ff0e84b6640 1 --2- 172.21.15.87:0/1877797060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0e0142fc0 0x7ff0e01f5590 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:37.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.073+0000 7ff0e84b6640 1 --2- 172.21.15.87:0/1877797060 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7ff0e00b5ad0 0x7ff0e0142a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:37.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.073+0000 7ff0e84b6640 1 --2- 172.21.15.87:0/1877797060 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7ff0cc003a30 0x7ff0e01f18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:37.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.073+0000 7ff0e84b6640 1 -- 172.21.15.87:0/1877797060 >> 172.21.15.87:0/1877797060 conn(0x7ff0e00b20f0 msgr2=0x7ff0e00706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:37.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.073+0000 7ff0e84b6640 1 -- 172.21.15.87:0/1877797060 shutdown_connections 2024-09-17T03:16:37.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.073+0000 7ff0e84b6640 1 -- 172.21.15.87:0/1877797060 wait complete. 2024-09-17T03:16:37.084 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-09-17T03:16:35.452382+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":230844,"kb_used_data":6280,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754743876,"statfs":{"total":773094113280,"available":772857729024,"internally_reserved":0,"allocated":6430720,"data_stored":3342872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002938"},"pg_stats":[{"pgid":"1.0","version":"55'74","reported_seq":180,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-17T03:16:14.000032+0000","last_change":"2024-09-17T03:16:14.000032+0000","last_active":"2024-09-17T03:16:14.000032+0000","last_peered":"2024-09-17T03:16:14.000032+0000","last_clean":"2024-09-17T03:16:14.000032+0000","last_became_active":"2024-09-17T03:14:04.861731+0000","last_became_peered":"2024-09-17T03:14:04.861731+0000","last_unstale":"2024-09-17T03:16:14.000032+0000","last_undegraded":"2024-09-17T03:16:14.000032+0000","last_fullsized":"2024-09-17T03:16:14.000032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"55'74","last_scrub_stamp":"2024-09-17T03:16:13.999944+0000","last_deep_scrub":"55'74","last_deep_scrub_stamp":"2024-09-17T03:16:13.999944+0000","last_clean_scrub_stamp":"2024-09-17T03:16:13.999944+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-18T04:00:11.551461+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509475,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27672,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":655360,"data_stored":270267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509475,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27676,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":655360,"data_stored":270267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509474,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1249280,"data_stored":860635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509475,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27672,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":655360,"data_stored":270267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509475,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27672,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":655360,"data_stored":270267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509475,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28244,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343596,"statfs":{"total":96636764160,"available":96607842304,"internally_reserved":0,"allocated":1249280,"data_stored":860635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509474,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27672,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":655360,"data_stored":270267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31832,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":655360,"data_stored":270267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-17T03:16:37.085 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-17T03:16:37.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.201+0000 7fb868de4640 1 -- 172.21.15.87:0/407798281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb850003c30 msgr2=0x7fb850004080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:37.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.201+0000 7fb868de4640 1 --2- 172.21.15.87:0/407798281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb850003c30 0x7fb850004080 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00c4a0 tx=0x7fb850005830 comp rx=0 tx=0).stop 2024-09-17T03:16:37.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.201+0000 7fb868de4640 1 -- 172.21.15.87:0/407798281 shutdown_connections 2024-09-17T03:16:37.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.201+0000 7fb868de4640 1 --2- 172.21.15.87:0/407798281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb850003c30 0x7fb850004080 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:37.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.201+0000 7fb868de4640 1 -- 172.21.15.87:0/407798281 >> 172.21.15.87:0/407798281 conn(0x7fb8640b1c40 msgr2=0x7fb8640b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:37.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.201+0000 7fb868de4640 1 -- 172.21.15.87:0/407798281 shutdown_connections 2024-09-17T03:16:37.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.201+0000 7fb868de4640 1 -- 172.21.15.87:0/407798281 wait complete. 2024-09-17T03:16:37.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.203+0000 7fb868de4640 1 Processor -- start 2024-09-17T03:16:37.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.203+0000 7fb868de4640 1 -- start start 2024-09-17T03:16:37.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.204+0000 7fb868de4640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb850003c30 0x7fb86414fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:37.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.204+0000 7fb868de4640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8641f9210 0x7fb864143ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:37.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.204+0000 7fb868de4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8641f8600 0x7fb8641f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:37.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.204+0000 7fb868de4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb86415ef40 con 0x7fb8641f8600 2024-09-17T03:16:37.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.204+0000 7fb868de4640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7fb8640732e0 con 0x7fb8641f9210 2024-09-17T03:16:37.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.204+0000 7fb868de4640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7fb86415f0c0 con 0x7fb850003c30 2024-09-17T03:16:37.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.204+0000 7fb862d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8641f8600 0x7fb8641f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:37.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.204+0000 7fb862d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8641f8600 0x7fb8641f8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59402/0 (socket says 172.21.15.87:59402) 2024-09-17T03:16:37.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.204+0000 7fb862d9d640 1 -- 172.21.15.87:0/2039105465 learned_addr learned my addr 172.21.15.87:0/2039105465 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:37.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.205+0000 7fb861d9b640 1 --2- 172.21.15.87:0/2039105465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8641f9210 0x7fb864143ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:37.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.205+0000 7fb86259c640 1 --2- 172.21.15.87:0/2039105465 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb850003c30 0x7fb86414fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:37.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.205+0000 7fb862d9d640 1 -- 172.21.15.87:0/2039105465 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb850003c30 msgr2=0x7fb86414fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:37.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.205+0000 7fb862d9d640 1 --2- 172.21.15.87:0/2039105465 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb850003c30 0x7fb86414fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:37.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.205+0000 7fb862d9d640 1 -- 172.21.15.87:0/2039105465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8641f9210 msgr2=0x7fb864143ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:37.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.205+0000 7fb862d9d640 1 --2- 172.21.15.87:0/2039105465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8641f9210 0x7fb864143ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:37.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.205+0000 7fb862d9d640 1 -- 172.21.15.87:0/2039105465 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8641f8f50 con 0x7fb8641f8600 2024-09-17T03:16:37.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.205+0000 7fb862d9d640 1 --2- 172.21.15.87:0/2039105465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8641f8600 0x7fb8641f8a10 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb86415c9a0 tx=0x7fb85c010e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:37.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.205+0000 7fb85affd640 1 -- 172.21.15.87:0/2039105465 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c00edb0 con 0x7fb8641f8600 2024-09-17T03:16:37.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.205+0000 7fb85affd640 1 -- 172.21.15.87:0/2039105465 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb85c015070 con 0x7fb8641f8600 2024-09-17T03:16:37.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.205+0000 7fb85affd640 1 -- 172.21.15.87:0/2039105465 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c003bd0 con 0x7fb8641f8600 2024-09-17T03:16:37.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.205+0000 7fb868de4640 1 -- 172.21.15.87:0/2039105465 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb864150770 con 0x7fb8641f8600 2024-09-17T03:16:37.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.205+0000 7fb868de4640 1 -- 172.21.15.87:0/2039105465 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb864150d30 con 0x7fb8641f8600 2024-09-17T03:16:37.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.206+0000 7fb868de4640 1 -- 172.21.15.87:0/2039105465 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb86415f620 con 0x7fb8641f8600 2024-09-17T03:16:37.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.208+0000 7fb85affd640 1 -- 172.21.15.87:0/2039105465 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb85c004030 con 0x7fb8641f8600 2024-09-17T03:16:37.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.208+0000 7fb85affd640 1 --2- 172.21.15.87:0/2039105465 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb844078b90 0x7fb84407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:37.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.208+0000 7fb85affd640 1 -- 172.21.15.87:0/2039105465 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7fb85c09b4c0 con 0x7fb8641f8600 2024-09-17T03:16:37.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.208+0000 7fb86259c640 1 --2- 172.21.15.87:0/2039105465 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb844078b90 0x7fb84407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:37.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.209+0000 7fb86259c640 1 --2- 172.21.15.87:0/2039105465 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb844078b90 0x7fb84407b0a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb84c033000 tx=0x7fb84c02d070 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:37.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.212+0000 7fb85affd640 1 -- 172.21.15.87:0/2039105465 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7fb8641f8f50 con 0x7fb8641f8600 2024-09-17T03:16:37.389 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.390+0000 7fb868de4640 1 -- 172.21.15.87:0/2039105465 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fb86415c780 con 0x7fb8641f8600 2024-09-17T03:16:37.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.394+0000 7fb85affd640 1 -- 172.21.15.87:0/2039105465 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v36)=0 v36) ==== 123+0+0 (secure 0 0 0) 0x7fb85c0635b0 con 0x7fb8641f8600 2024-09-17T03:16:37.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.395+0000 7fb85affd640 1 -- 172.21.15.87:0/2039105465 <== mon.0 v2:172.21.15.87:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7fb85c0630c0 con 0x7fb8641f8600 2024-09-17T03:16:37.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.396+0000 7fb868de4640 1 -- 172.21.15.87:0/2039105465 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb844078b90 msgr2=0x7fb84407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:37.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.396+0000 7fb868de4640 1 --2- 172.21.15.87:0/2039105465 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb844078b90 0x7fb84407b0a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb84c033000 tx=0x7fb84c02d070 comp rx=0 tx=0).stop 2024-09-17T03:16:37.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.396+0000 7fb868de4640 1 -- 172.21.15.87:0/2039105465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8641f8600 msgr2=0x7fb8641f8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:37.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.396+0000 7fb868de4640 1 --2- 172.21.15.87:0/2039105465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8641f8600 0x7fb8641f8a10 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb86415c9a0 tx=0x7fb85c010e80 comp rx=0 tx=0).stop 2024-09-17T03:16:37.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.397+0000 7fb868de4640 1 -- 172.21.15.87:0/2039105465 shutdown_connections 2024-09-17T03:16:37.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.397+0000 7fb868de4640 1 --2- 172.21.15.87:0/2039105465 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7fb844078b90 0x7fb84407b0a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:37.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.397+0000 7fb868de4640 1 --2- 172.21.15.87:0/2039105465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8641f8600 0x7fb8641f8a10 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:37.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.397+0000 7fb868de4640 1 --2- 172.21.15.87:0/2039105465 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7fb8641f9210 0x7fb864143ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:37.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.397+0000 7fb868de4640 1 --2- 172.21.15.87:0/2039105465 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7fb850003c30 0x7fb86414fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:37.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.397+0000 7fb868de4640 1 -- 172.21.15.87:0/2039105465 >> 172.21.15.87:0/2039105465 conn(0x7fb8640b1c40 msgr2=0x7fb86415e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:37.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.397+0000 7fb868de4640 1 -- 172.21.15.87:0/2039105465 shutdown_connections 2024-09-17T03:16:37.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.397+0000 7fb868de4640 1 -- 172.21.15.87:0/2039105465 wait complete. 2024-09-17T03:16:37.407 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-17T03:16:37.407 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-17T03:16:37.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.524+0000 7f0d52207640 1 -- 172.21.15.87:0/462898800 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d34004d90 msgr2=0x7f0d34005240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.524+0000 7f0d52207640 1 --2- 172.21.15.87:0/462898800 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d34004d90 0x7f0d34005240 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c0cb710 tx=0x7f0d34005830 comp rx=0 tx=0).stop 2024-09-17T03:16:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.524+0000 7f0d4bfff640 1 -- 172.21.15.87:0/462898800 reap_dead start 2024-09-17T03:16:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.524+0000 7f0d52207640 1 -- 172.21.15.87:0/462898800 shutdown_connections 2024-09-17T03:16:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.524+0000 7f0d52207640 1 -- 172.21.15.87:0/462898800 >> 172.21.15.87:0/462898800 conn(0x7f0d4c0d1110 msgr2=0x7f0d4c0d1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.524+0000 7f0d52207640 1 -- 172.21.15.87:0/462898800 shutdown_connections 2024-09-17T03:16:37.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.524+0000 7f0d52207640 1 -- 172.21.15.87:0/462898800 wait complete. 2024-09-17T03:16:37.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.526+0000 7f0d52207640 1 Processor -- start 2024-09-17T03:16:37.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.526+0000 7f0d52207640 1 -- start start 2024-09-17T03:16:37.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.527+0000 7f0d52207640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d34004d90 0x7f0d4c0b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:37.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.528+0000 7f0d52207640 1 --2- >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0d4c1ef850 0x7f0d4c1f8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:37.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.528+0000 7f0d52207640 1 --2- >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d4c1f9520 0x7f0d4c0b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:37.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.528+0000 7f0d52207640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c15d240 con 0x7f0d34004d90 2024-09-17T03:16:37.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.528+0000 7f0d52207640 1 -- --> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c15a670 con 0x7f0d4c1f9520 2024-09-17T03:16:37.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.528+0000 7f0d52207640 1 -- --> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] -- mon_getmap magic: 0 -- 0x7f0d4c15d3c0 con 0x7f0d4c1ef850 2024-09-17T03:16:37.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.528+0000 7f0d4b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d34004d90 0x7f0d4c0b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:37.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.528+0000 7f0d4b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d34004d90 0x7f0d4c0b5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59418/0 (socket says 172.21.15.87:59418) 2024-09-17T03:16:37.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.528+0000 7f0d4b7fe640 1 -- 172.21.15.87:0/1349601114 learned_addr learned my addr 172.21.15.87:0/1349601114 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-09-17T03:16:37.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.528+0000 7f0d4affd640 1 --2- 172.21.15.87:0/1349601114 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0d4c1ef850 0x7f0d4c1f8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:37.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.528+0000 7f0d4bfff640 1 --2- 172.21.15.87:0/1349601114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d4c1f9520 0x7f0d4c0b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:37.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.528+0000 7f0d4b7fe640 1 -- 172.21.15.87:0/1349601114 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0d4c1ef850 msgr2=0x7f0d4c1f8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:37.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.528+0000 7f0d4b7fe640 1 --2- 172.21.15.87:0/1349601114 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0d4c1ef850 0x7f0d4c1f8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:37.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.528+0000 7f0d4b7fe640 1 -- 172.21.15.87:0/1349601114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d4c1f9520 msgr2=0x7f0d4c0b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:37.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.528+0000 7f0d4b7fe640 1 --2- 172.21.15.87:0/1349601114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d4c1f9520 0x7f0d4c0b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:37.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.528+0000 7f0d4b7fe640 1 -- 172.21.15.87:0/1349601114 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4c0c9930 con 0x7f0d34004d90 2024-09-17T03:16:37.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.528+0000 7f0d4b7fe640 1 --2- 172.21.15.87:0/1349601114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d34004d90 0x7f0d4c0b5d20 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c006630 tx=0x7f0d3c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:37.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.528+0000 7f0d33fff640 1 -- 172.21.15.87:0/1349601114 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c015070 con 0x7f0d34004d90 2024-09-17T03:16:37.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.529+0000 7f0d33fff640 1 -- 172.21.15.87:0/1349601114 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f0d3c011950 con 0x7f0d34004d90 2024-09-17T03:16:37.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.529+0000 7f0d33fff640 1 -- 172.21.15.87:0/1349601114 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c011dc0 con 0x7f0d34004d90 2024-09-17T03:16:37.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.529+0000 7f0d52207640 1 -- 172.21.15.87:0/1349601114 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4c0c9c10 con 0x7f0d34004d90 2024-09-17T03:16:37.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.529+0000 7f0d52207640 1 -- 172.21.15.87:0/1349601114 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d4c0ca1d0 con 0x7f0d34004d90 2024-09-17T03:16:37.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.530+0000 7f0d52207640 1 -- 172.21.15.87:0/1349601114 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d4c15eb90 con 0x7f0d34004d90 2024-09-17T03:16:37.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.531+0000 7f0d33fff640 1 -- 172.21.15.87:0/1349601114 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0d3c018070 con 0x7f0d34004d90 2024-09-17T03:16:37.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.531+0000 7f0d33fff640 1 --2- 172.21.15.87:0/1349601114 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0d38078b90 0x7f0d3807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-17T03:16:37.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.531+0000 7f0d33fff640 1 -- 172.21.15.87:0/1349601114 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f0d3c09b770 con 0x7f0d34004d90 2024-09-17T03:16:37.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.532+0000 7f0d4affd640 1 --2- 172.21.15.87:0/1349601114 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0d38078b90 0x7f0d3807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-17T03:16:37.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.532+0000 7f0d4affd640 1 --2- 172.21.15.87:0/1349601114 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0d38078b90 0x7f0d3807b0a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c0b57e0 tx=0x7f0d40005f90 comp rx=0 tx=0).ready entity=mgr.4117 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-17T03:16:37.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.536+0000 7f0d33fff640 1 -- 172.21.15.87:0/1349601114 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+195964 (secure 0 0 0) 0x7f0d3c063860 con 0x7f0d34004d90 2024-09-17T03:16:37.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.753+0000 7f0d52207640 1 -- 172.21.15.87:0/1349601114 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d4c0b7090 con 0x7f0d34004d90 2024-09-17T03:16:37.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.753+0000 7f0d33fff640 1 -- 172.21.15.87:0/1349601114 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+3917 (secure 0 0 0) 0x7f0d3c068290 con 0x7f0d34004d90 2024-09-17T03:16:37.753 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-09-17T03:16:37.753 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":22,"btime":"2024-09-17T03:16:06:565163+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5104,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6834/1606650259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1606650259},{"type":"v1","addr":"172.21.15.87:6834","nonce":1606650259}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":5116,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6836/3876173555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6834","nonce":3876173555},{"type":"v1","addr":"172.21.15.149:6836","nonce":3876173555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17},{"gid":5347,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.149:6837/1071878070","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6835","nonce":1071878070},{"type":"v1","addr":"172.21.15.149:6837","nonce":1071878070}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1657329632","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6833","nonce":1657329632},{"type":"v1","addr":"172.21.15.87:6835","nonce":1657329632}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":17}],"filesystems":[]} 2024-09-17T03:16:37.753 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 22 2024-09-17T03:16:37.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.755+0000 7f0d52207640 1 -- 172.21.15.87:0/1349601114 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0d38078b90 msgr2=0x7f0d3807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:37.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.756+0000 7f0d52207640 1 --2- 172.21.15.87:0/1349601114 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0d38078b90 0x7f0d3807b0a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c0b57e0 tx=0x7f0d40005f90 comp rx=0 tx=0).stop 2024-09-17T03:16:37.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.756+0000 7f0d52207640 1 -- 172.21.15.87:0/1349601114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d34004d90 msgr2=0x7f0d4c0b5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-17T03:16:37.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.756+0000 7f0d52207640 1 --2- 172.21.15.87:0/1349601114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d34004d90 0x7f0d4c0b5d20 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c006630 tx=0x7f0d3c0039c0 comp rx=0 tx=0).stop 2024-09-17T03:16:37.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.756+0000 7f0d52207640 1 -- 172.21.15.87:0/1349601114 shutdown_connections 2024-09-17T03:16:37.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.756+0000 7f0d52207640 1 --2- 172.21.15.87:0/1349601114 >> [v2:172.21.15.149:6800/973360146,v1:172.21.15.149:6801/973360146] conn(0x7f0d38078b90 0x7f0d3807b0a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:37.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.756+0000 7f0d52207640 1 --2- 172.21.15.87:0/1349601114 >> [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] conn(0x7f0d4c1f9520 0x7f0d4c0b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:37.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.756+0000 7f0d52207640 1 --2- 172.21.15.87:0/1349601114 >> [v2:172.21.15.149:3301/0,v1:172.21.15.149:6790/0] conn(0x7f0d4c1ef850 0x7f0d4c1f8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:37.755 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.756+0000 7f0d52207640 1 --2- 172.21.15.87:0/1349601114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d34004d90 0x7f0d4c0b5d20 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-17T03:16:37.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.756+0000 7f0d52207640 1 -- 172.21.15.87:0/1349601114 >> 172.21.15.87:0/1349601114 conn(0x7f0d4c0d1110 msgr2=0x7f0d4c0cf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-17T03:16:37.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.756+0000 7f0d52207640 1 -- 172.21.15.87:0/1349601114 shutdown_connections 2024-09-17T03:16:37.756 INFO:teuthology.orchestra.run.smithi087.stderr:2024-09-17T03:16:37.756+0000 7f0d52207640 1 -- 172.21.15.87:0/1349601114 wait complete. 2024-09-17T03:16:37.766 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-17T03:16:37.766 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-17T03:16:37.766 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-17T03:16:37.767 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:16:37.776 INFO:tasks.ceph.mds.a:Stopped 2024-09-17T03:16:37.777 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-17T03:16:37.777 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:16:37.828 INFO:tasks.ceph.mds.c:Stopped 2024-09-17T03:16:37.828 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-17T03:16:37.828 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:16:37.843 INFO:tasks.ceph.mds.b:Stopped 2024-09-17T03:16:37.843 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-17T03:16:37.843 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:16:37.895 INFO:tasks.ceph.mds.d:Stopped 2024-09-17T03:16:37.895 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-17T03:16:37.895 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-17T03:16:37.896 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:16:37.936 INFO:tasks.ceph.osd.0:Stopped 2024-09-17T03:16:37.936 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-17T03:16:37.936 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:16:38.010 INFO:tasks.ceph.osd.1:Stopped 2024-09-17T03:16:38.011 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-17T03:16:38.011 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:16:38.096 INFO:tasks.ceph.osd.2:Stopped 2024-09-17T03:16:38.096 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-17T03:16:38.096 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:16:38.097 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-17T03:16:38.097 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-09-17T03:16:38.097 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-09-17T03:16:38.097 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-09-17T03:16:38.124 INFO:tasks.ceph.osd.3:Stopped 2024-09-17T03:16:38.125 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-17T03:16:38.125 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:16:38.159 INFO:tasks.ceph.osd.4:Stopped 2024-09-17T03:16:38.159 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-17T03:16:38.160 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:16:38.240 INFO:tasks.ceph.osd.5:Stopped 2024-09-17T03:16:38.240 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-17T03:16:38.240 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:16:38.308 INFO:tasks.ceph.osd.6:Stopped 2024-09-17T03:16:38.309 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-17T03:16:38.309 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:16:38.376 INFO:tasks.ceph.osd.7:Stopped 2024-09-17T03:16:38.377 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-17T03:16:38.378 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-17T03:16:38.378 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:16:38.407 INFO:tasks.ceph.mgr.y:Stopped 2024-09-17T03:16:38.407 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-17T03:16:38.407 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:16:38.445 INFO:tasks.ceph.mgr.x:Stopped 2024-09-17T03:16:38.445 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-17T03:16:38.446 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-17T03:16:38.446 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:16:38.459 INFO:tasks.ceph.mon.a:Stopped 2024-09-17T03:16:38.459 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-17T03:16:38.459 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:16:38.497 INFO:tasks.ceph.mon.b:Stopped 2024-09-17T03:16:38.498 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-17T03:16:38.498 INFO:teuthology.orchestra.run:waiting for 300 2024-09-17T03:16:38.550 INFO:tasks.ceph.mon.c:Stopped 2024-09-17T03:16:38.550 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-17T03:16:38.551 DEBUG:teuthology.orchestra.run.smithi087:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-17T03:16:38.584 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi087.front.sepia.ceph.com 2024-09-17T03:16:38.584 DEBUG:teuthology.orchestra.run.smithi087:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-17T03:16:38.787 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi087.front.sepia.ceph.com 2024-09-17T03:16:38.788 DEBUG:teuthology.orchestra.run.smithi087:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-17T03:16:38.866 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi087.front.sepia.ceph.com 2024-09-17T03:16:38.866 DEBUG:teuthology.orchestra.run.smithi087:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-17T03:16:38.923 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi087.front.sepia.ceph.com 2024-09-17T03:16:38.923 DEBUG:teuthology.orchestra.run.smithi087:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-17T03:16:38.973 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi149.front.sepia.ceph.com 2024-09-17T03:16:38.973 DEBUG:teuthology.orchestra.run.smithi149:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-17T03:16:39.152 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi149.front.sepia.ceph.com 2024-09-17T03:16:39.152 DEBUG:teuthology.orchestra.run.smithi149:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-17T03:16:39.215 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi149.front.sepia.ceph.com 2024-09-17T03:16:39.215 DEBUG:teuthology.orchestra.run.smithi149:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-17T03:16:39.273 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi149.front.sepia.ceph.com 2024-09-17T03:16:39.273 DEBUG:teuthology.orchestra.run.smithi149:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-17T03:16:39.323 INFO:tasks.ceph:Archiving mon data... 2024-09-17T03:16:39.325 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908414/data/mon.a.tgz 2024-09-17T03:16:39.325 DEBUG:teuthology.orchestra.run.smithi087:> mktemp 2024-09-17T03:16:39.341 INFO:teuthology.orchestra.run.smithi087.stdout:/tmp/tmp.a3zXzEag7L 2024-09-17T03:16:39.342 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.a3zXzEag7L 2024-09-17T03:16:39.367 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:16:39.385 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:16:39.500 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0666 /tmp/tmp.a3zXzEag7L 2024-09-17T03:16:39.584 DEBUG:teuthology.orchestra.remote:smithi087:/tmp/tmp.a3zXzEag7L is 345KB 2024-09-17T03:16:39.680 DEBUG:teuthology.orchestra.run.smithi087:> rm -fr /tmp/tmp.a3zXzEag7L 2024-09-17T03:16:39.695 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908414/data/mon.b.tgz 2024-09-17T03:16:39.695 DEBUG:teuthology.orchestra.run.smithi149:> mktemp 2024-09-17T03:16:39.712 INFO:teuthology.orchestra.run.smithi149.stdout:/tmp/tmp.H5C7Mczpp9 2024-09-17T03:16:39.712 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.H5C7Mczpp9 2024-09-17T03:16:39.873 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 0666 /tmp/tmp.H5C7Mczpp9 2024-09-17T03:16:39.957 DEBUG:teuthology.orchestra.remote:smithi149:/tmp/tmp.H5C7Mczpp9 is 460KB 2024-09-17T03:16:40.064 DEBUG:teuthology.orchestra.run.smithi149:> rm -fr /tmp/tmp.H5C7Mczpp9 2024-09-17T03:16:40.080 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908414/data/mon.c.tgz 2024-09-17T03:16:40.080 DEBUG:teuthology.orchestra.run.smithi149:> mktemp 2024-09-17T03:16:40.134 INFO:teuthology.orchestra.run.smithi149.stdout:/tmp/tmp.wrCX3TL7oR 2024-09-17T03:16:40.135 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.wrCX3TL7oR 2024-09-17T03:16:40.283 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 0666 /tmp/tmp.wrCX3TL7oR 2024-09-17T03:16:40.368 DEBUG:teuthology.orchestra.remote:smithi149:/tmp/tmp.wrCX3TL7oR is 460KB 2024-09-17T03:16:40.467 DEBUG:teuthology.orchestra.run.smithi149:> rm -fr /tmp/tmp.wrCX3TL7oR 2024-09-17T03:16:40.482 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-17T03:16:40.482 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T03:16:40.485 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-17T03:16:40.569 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T03:16:40.602 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-17T03:16:40.602 INFO:tasks.ceph:Archiving crash dumps... 2024-09-17T03:16:40.603 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908414/remote/smithi087/crash 2024-09-17T03:16:40.604 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T03:16:40.637 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908414/remote/smithi149/crash 2024-09-17T03:16:40.638 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-17T03:16:40.666 INFO:tasks.ceph:Shutting down logrotate 2024-09-17T03:16:40.667 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:16:40.682 DEBUG:teuthology.orchestra.run.smithi149:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-17T03:16:40.753 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-17T03:16:40.779 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-17T03:16:40.804 INFO:tasks.ceph:Compressing logs... 2024-09-17T03:16:40.804 DEBUG:teuthology.orchestra.run.smithi087:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T03:16:40.822 DEBUG:teuthology.orchestra.run.smithi149:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-17T03:16:40.845 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-17T03:16:40.845 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42943.log 2024-09-17T03:16:40.864 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45127.log 2024-09-17T03:16:40.864 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.42943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37700.log 2024-09-17T03:16:40.864 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42943.log.gz 2024-09-17T03:16:40.865 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48760.log 2024-09-17T03:16:40.865 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45127.log.gz 2024-09-17T03:16:40.865 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44871.log 2024-09-17T03:16:40.865 INFO:teuthology.orchestra.run.smithi087.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.37700.log.gz 2024-09-17T03:16:40.865 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.48760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45421.log 2024-09-17T03:16:40.865 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37058.log 2024-09-17T03:16:40.865 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48760.log.gz 2024-09-17T03:16:40.865 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44871.log.gz/var/log/ceph/ceph-client.admin.45421.log: 2024-09-17T03:16:40.866 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48493.log 2024-09-17T03:16:40.866 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49096.log 2024-09-17T03:16:40.866 INFO:teuthology.orchestra.run.smithi087.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.45421.log.gz 2024-09-17T03:16:40.866 INFO:teuthology.orchestra.run.smithi087.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.37058.log.gz 2024-09-17T03:16:40.866 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.48493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47680.log 2024-09-17T03:16:40.866 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48493.log.gz 2024-09-17T03:16:40.866 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45211.log 2024-09-17T03:16:40.866 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49096.log.gz 2024-09-17T03:16:40.867 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.47680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55465.log 2024-09-17T03:16:40.867 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47680.log.gz 2024-09-17T03:16:40.867 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49749.log 2024-09-17T03:16:40.867 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55465.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45211.log.gz 2024-09-17T03:16:40.867 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46718.log 2024-09-17T03:16:40.867 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55465.log.gz 2024-09-17T03:16:40.867 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44257.log 2024-09-17T03:16:40.867 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6%/var/log/ceph/ceph-client.admin.46718.log: -- replaced with /var/log/ceph/ceph-client.admin.49749.log.gz 2024-09-17T03:16:40.868 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43304.log 2024-09-17T03:16:40.868 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6%/var/log/ceph/ceph-client.admin.44257.log: -- replaced with /var/log/ceph/ceph-client.admin.46718.log.gz 2024-09-17T03:16:40.868 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43024.log 2024-09-17T03:16:40.868 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44257.log.gz 2024-09-17T03:16:40.868 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43573.log 2024-09-17T03:16:40.868 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4%/var/log/ceph/ceph-client.admin.43024.log: -- replaced with /var/log/ceph/ceph-client.admin.43304.log.gz 2024-09-17T03:16:40.868 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52537.log 2024-09-17T03:16:40.868 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43024.log.gz 2024-09-17T03:16:40.868 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51900.log 2024-09-17T03:16:40.869 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43573.log.gz 2024-09-17T03:16:40.869 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36622.log 2024-09-17T03:16:40.869 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52537.log.gz 2024-09-17T03:16:40.869 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56184.log 2024-09-17T03:16:40.869 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51900.log.gz 2024-09-17T03:16:40.869 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47083.log 2024-09-17T03:16:40.869 INFO:teuthology.orchestra.run.smithi087.stderr: 79.4% -- replaced with /var/log/ceph/ceph-client.admin.36622.log.gz 2024-09-17T03:16:40.869 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.56184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47470.log 2024-09-17T03:16:40.870 INFO:teuthology.orchestra.run.smithi087.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56184.log.gz 2024-09-17T03:16:40.870 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.47083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56142.log 2024-09-17T03:16:40.870 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.47470.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47083.log.gz 2024-09-17T03:16:40.870 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44593.log 2024-09-17T03:16:40.870 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47470.log.gz 2024-09-17T03:16:40.870 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56058.log 2024-09-17T03:16:40.870 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.56142.log: /var/log/ceph/ceph-client.admin.44593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51419.log 2024-09-17T03:16:40.870 INFO:teuthology.orchestra.run.smithi087.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56142.log.gz 2024-09-17T03:16:40.870 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3%/var/log/ceph/ceph-client.admin.56058.log: -- replaced with /var/log/ceph/ceph-client.admin.44593.log.gz 2024-09-17T03:16:40.871 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-17T03:16:40.871 INFO:teuthology.orchestra.run.smithi087.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.56058.log.gz 2024-09-17T03:16:40.871 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54983.log 2024-09-17T03:16:40.871 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51419.log.gz 2024-09-17T03:16:40.871 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52152.log 2024-09-17T03:16:40.871 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43825.log 2024-09-17T03:16:40.871 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54983.log.gz 2024-09-17T03:16:40.871 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36845.log 2024-09-17T03:16:40.872 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52152.log.gz 2024-09-17T03:16:40.872 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45253.log 2024-09-17T03:16:40.872 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43825.log: /var/log/ceph/ceph-client.admin.36845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56268.log 87.6% 2024-09-17T03:16:40.872 INFO:teuthology.orchestra.run.smithi087.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43825.log.gz 2024-09-17T03:16:40.872 INFO:teuthology.orchestra.run.smithi087.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.36845.log.gz 2024-09-17T03:16:40.872 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47909.log 2024-09-17T03:16:40.872 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.56268.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45253.log.gz 2024-09-17T03:16:40.872 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49506.log 2024-09-17T03:16:40.873 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56268.log.gz 2024-09-17T03:16:40.873 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.47909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56100.log 2024-09-17T03:16:40.873 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47909.log.gz 2024-09-17T03:16:40.873 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.49506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56646.log 2024-09-17T03:16:40.873 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.56100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52451.log 2024-09-17T03:16:40.873 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56100.log.gz 2024-09-17T03:16:40.873 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.56646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46844.log 2024-09-17T03:16:40.873 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52451.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56646.log.gz 2024-09-17T03:16:40.873 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43485.log 2024-09-17T03:16:40.874 INFO:teuthology.orchestra.run.smithi087.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.52451.log.gz 2024-09-17T03:16:40.874 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.46844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47141.log 2024-09-17T03:16:40.874 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46844.log.gz 2024-09-17T03:16:40.874 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44591.log 2024-09-17T03:16:40.874 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43485.log: 91.1% -- replaced with /var/log/ceph/ceph-client.0.49506.log.gz 2024-09-17T03:16:40.874 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.47141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52851.log 2024-09-17T03:16:40.874 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43485.log.gz 2024-09-17T03:16:40.874 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47141.log.gz 2024-09-17T03:16:40.875 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44589.log 2024-09-17T03:16:40.875 INFO:teuthology.orchestra.run.smithi087.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44591.log.gz 2024-09-17T03:16:40.875 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37610.log 2024-09-17T03:16:40.875 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52851.log: /var/log/ceph/ceph-client.admin.44589.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52851.log.gz 2024-09-17T03:16:40.875 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43741.log 2024-09-17T03:16:40.875 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44589.log.gz 2024-09-17T03:16:40.875 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52767.log 2024-09-17T03:16:40.875 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37610.log: /var/log/ceph/ceph-client.admin.43741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56436.log 2024-09-17T03:16:40.876 INFO:teuthology.orchestra.run.smithi087.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.37610.log.gz 2024-09-17T03:16:40.876 INFO:teuthology.orchestra.run.smithi087.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.43741.log.gz 2024-09-17T03:16:40.876 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48323.log 2024-09-17T03:16:40.876 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4%/var/log/ceph/ceph-client.admin.56436.log: -- replaced with /var/log/ceph/ceph-client.admin.52767.log.gz 2024-09-17T03:16:40.876 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55806.log 2024-09-17T03:16:40.876 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56436.log.gz 2024-09-17T03:16:40.876 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.48323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44258.log 2024-09-17T03:16:40.876 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48323.log.gz 2024-09-17T03:16:40.877 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36962.log 2024-09-17T03:16:40.877 INFO:teuthology.orchestra.run.smithi087.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55806.log.gz 2024-09-17T03:16:40.877 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51545.log 2024-09-17T03:16:40.877 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44258.log.gz 2024-09-17T03:16:40.877 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48588.log 2024-09-17T03:16:40.877 INFO:teuthology.orchestra.run.smithi087.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36962.log.gz 2024-09-17T03:16:40.877 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46068.log 2024-09-17T03:16:40.877 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51545.log.gz 2024-09-17T03:16:40.878 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.48588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43993.log 2024-09-17T03:16:40.878 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48588.log.gz 2024-09-17T03:16:40.878 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.46068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36732.log 2024-09-17T03:16:40.878 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46068.log.gz 2024-09-17T03:16:40.878 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51671.log 2024-09-17T03:16:40.878 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43993.log.gz 2024-09-17T03:16:40.878 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53923.log 2024-09-17T03:16:40.878 INFO:teuthology.orchestra.run.smithi087.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36732.log.gz 2024-09-17T03:16:40.878 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47951.log 2024-09-17T03:16:40.879 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51671.log.gz 2024-09-17T03:16:40.879 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51503.log 2024-09-17T03:16:40.879 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53923.log.gz 2024-09-17T03:16:40.879 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.30382.log 2024-09-17T03:16:40.892 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36669.log 2024-09-17T03:16:40.893 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-17T03:16:40.893 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-17T03:16:40.893 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-client.admin.36669.log: /var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36629.log 2024-09-17T03:16:40.893 INFO:teuthology.orchestra.run.smithi149.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36669.log.gz 2024-09-17T03:16:40.893 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36909.log 2024-09-17T03:16:40.893 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-client.admin.36629.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-17T03:16:40.893 INFO:teuthology.orchestra.run.smithi149.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36629.log.gz 2024-09-17T03:16:40.893 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-client.admin.36909.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T03:16:40.894 INFO:teuthology.orchestra.run.smithi149.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36909.log.gz 2024-09-17T03:16:40.894 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.42829.log 2024-09-17T03:16:40.894 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36869.log 2024-09-17T03:16:40.894 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-client.3.42829.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-17T03:16:40.894 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-client.admin.36869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36709.log 2024-09-17T03:16:40.894 INFO:teuthology.orchestra.run.smithi149.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36869.log.gz 2024-09-17T03:16:40.894 INFO:teuthology.orchestra.run.smithi149.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.3.42829.log.gz 2024-09-17T03:16:40.894 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42372.log 2024-09-17T03:16:40.895 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.36709.log: /var/log/ceph/ceph-osd.6.log 2024-09-17T03:16:40.895 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-client.1.42372.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36709.log.gz 2024-09-17T03:16:40.895 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-17T03:16:40.895 INFO:teuthology.orchestra.run.smithi149.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T03:16:40.895 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T03:16:40.895 INFO:teuthology.orchestra.run.smithi149.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.1.42372.log.gz 2024-09-17T03:16:40.895 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-17T03:16:40.895 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-17T03:16:40.895 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-mds.d.log: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T03:16:40.896 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36749.log 2024-09-17T03:16:40.896 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-mgr.x.log: 92.0% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-17T03:16:40.896 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36829.log 2024-09-17T03:16:40.896 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-client.admin.36749.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36749.log.gz 2024-09-17T03:16:40.904 INFO:teuthology.orchestra.run.smithi149.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-17T03:16:40.904 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-17T03:16:40.905 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-client.admin.36829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36789.log 2024-09-17T03:16:40.905 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-mon.b.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36829.log.gz 2024-09-17T03:16:40.921 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-client.admin.36789.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36789.log.gz 2024-09-17T03:16:40.927 INFO:teuthology.orchestra.run.smithi149.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-17T03:16:40.935 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.47951.log: /var/log/ceph/ceph-client.admin.51503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48928.log 2024-09-17T03:16:40.935 INFO:teuthology.orchestra.run.smithi087.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47951.log.gz 2024-09-17T03:16:40.936 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph.tmp-client.admin.30382.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51503.log.gz 2024-09-17T03:16:40.936 INFO:teuthology.orchestra.run.smithi087.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.30382.log.gz 2024-09-17T03:16:40.936 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51247.log 2024-09-17T03:16:40.936 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.48928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36482.log 2024-09-17T03:16:40.936 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48928.log.gz 2024-09-17T03:16:40.936 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48239.log 2024-09-17T03:16:40.936 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51247.log.gz 2024-09-17T03:16:40.937 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50681.log 2024-09-17T03:16:40.937 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36482.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36482.log.gz 2024-09-17T03:16:40.937 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.48239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53103.log 2024-09-17T03:16:40.937 INFO:teuthology.orchestra.run.smithi087.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48239.log.gz 2024-09-17T03:16:40.937 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45959.log 2024-09-17T03:16:40.937 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50681.log.gz 2024-09-17T03:16:40.937 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52809.log 2024-09-17T03:16:40.937 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53103.log: /var/log/ceph/ceph-client.0.45959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56226.log 2024-09-17T03:16:40.937 INFO:teuthology.orchestra.run.smithi087.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53103.log.gz 2024-09-17T03:16:40.938 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50001.log 2024-09-17T03:16:40.938 INFO:teuthology.orchestra.run.smithi087.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.0.45959.log.gz 87.5% 2024-09-17T03:16:40.938 INFO:teuthology.orchestra.run.smithi087.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52809.log.gz 2024-09-17T03:16:40.938 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.56226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36566.log 2024-09-17T03:16:40.938 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56226.log.gz 2024-09-17T03:16:40.938 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56310.log 2024-09-17T03:16:40.938 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50001.log.gz 2024-09-17T03:16:40.938 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37409.log 2024-09-17T03:16:40.939 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36566.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36566.log.gz 2024-09-17T03:16:40.939 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.56310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50983.log 2024-09-17T03:16:40.939 INFO:teuthology.orchestra.run.smithi087.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56310.log.gz 2024-09-17T03:16:40.939 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55423.log 2024-09-17T03:16:40.939 INFO:teuthology.orchestra.run.smithi087.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.37409.log.gz 2024-09-17T03:16:40.939 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.50983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52641.log 2024-09-17T03:16:40.939 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44594.log 2024-09-17T03:16:40.939 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55423.log: /var/log/ceph/ceph-client.admin.52641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48031.log 2024-09-17T03:16:40.940 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55423.log.gz 2024-09-17T03:16:40.940 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52641.log.gz 2024-09-17T03:16:40.940 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44259.log 2024-09-17T03:16:40.940 INFO:teuthology.orchestra.run.smithi087.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44594.log.gz 2024-09-17T03:16:40.940 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-17T03:16:40.940 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.48031.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.50983.log.gz 2024-09-17T03:16:40.940 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4%/var/log/ceph/ceph-client.admin.44259.log: -- replaced with /var/log/ceph/ceph-client.admin.48031.log.gz 2024-09-17T03:16:40.940 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46608.log 2024-09-17T03:16:40.940 INFO:teuthology.orchestra.run.smithi087.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44259.log.gz 2024-09-17T03:16:40.941 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50546.log 2024-09-17T03:16:40.941 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56771.log 2024-09-17T03:16:40.941 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.46608.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46608.log.gz 2024-09-17T03:16:40.941 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53145.log 2024-09-17T03:16:40.941 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50546.log.gz 2024-09-17T03:16:40.941 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49246.log 2024-09-17T03:16:40.941 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.56771.log: /var/log/ceph/ceph-client.admin.53145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36782.log 2024-09-17T03:16:40.941 INFO:teuthology.orchestra.run.smithi087.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56771.log.gz 2024-09-17T03:16:40.942 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53145.log.gz 2024-09-17T03:16:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43443.log 2024-09-17T03:16:40.942 INFO:teuthology.orchestra.run.smithi087.stderr: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-17T03:16:40.942 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49246.log.gz 2024-09-17T03:16:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48718.log 2024-09-17T03:16:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36782.log: /var/log/ceph/ceph-client.admin.43443.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36782.log.gz 2024-09-17T03:16:40.942 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55848.log 2024-09-17T03:16:40.942 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43443.log.gz 2024-09-17T03:16:40.943 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44867.log 2024-09-17T03:16:40.943 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.48718.log: /var/log/ceph/ceph-client.admin.55848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52026.log 2024-09-17T03:16:40.943 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48718.log.gz 2024-09-17T03:16:40.943 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55848.log.gz 2024-09-17T03:16:40.943 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37056.log 2024-09-17T03:16:40.943 INFO:teuthology.orchestra.run.smithi087.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.44867.log.gz 2024-09-17T03:16:40.943 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44931.log 2024-09-17T03:16:40.943 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52026.log.gz 2024-09-17T03:16:40.943 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43951.log 2024-09-17T03:16:40.944 INFO:teuthology.orchestra.run.smithi087.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.37056.log.gz 2024-09-17T03:16:40.944 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36999.log 2024-09-17T03:16:40.944 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44931.log.gz 2024-09-17T03:16:40.944 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45523.log 2024-09-17T03:16:40.944 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43951.log: /var/log/ceph/ceph-client.admin.36999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49943.log 2024-09-17T03:16:40.944 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43951.log.gz 2024-09-17T03:16:40.944 INFO:teuthology.orchestra.run.smithi087.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36999.log.gz 2024-09-17T03:16:40.944 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52327.log 2024-09-17T03:16:40.945 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45523.log.gz 2024-09-17T03:16:40.945 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48449.log 2024-09-17T03:16:40.945 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49943.log: /var/log/ceph/ceph-client.admin.52327.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49943.log.gz 2024-09-17T03:16:40.945 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36509.log 2024-09-17T03:16:40.945 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52327.log.gz 2024-09-17T03:16:40.945 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44595.log 2024-09-17T03:16:40.945 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.48449.log: /var/log/ceph/ceph-client.admin.36509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54810.log 2024-09-17T03:16:40.945 INFO:teuthology.orchestra.run.smithi087.stderr: 76.8% -- replaced with /var/log/ceph/ceph-client.admin.36509.log.gz 2024-09-17T03:16:40.945 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48449.log.gz 2024-09-17T03:16:40.946 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.46444.log 2024-09-17T03:16:40.946 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44595.log.gz 2024-09-17T03:16:40.946 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.0.54810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37047.log 2024-09-17T03:16:40.946 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43348.log 2024-09-17T03:16:40.946 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.2.46444.log: /var/log/ceph/ceph-client.admin.37047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55213.log 2024-09-17T03:16:40.946 INFO:teuthology.orchestra.run.smithi087.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.37047.log.gz 2024-09-17T03:16:40.946 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56394.log 2024-09-17T03:16:40.946 INFO:teuthology.orchestra.run.smithi087.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.0.54810.log.gz 2024-09-17T03:16:40.947 INFO:teuthology.orchestra.run.smithi087.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.2.46444.log.gz 2024-09-17T03:16:40.947 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43348.log.gz 2024-09-17T03:16:40.947 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44035.log 2024-09-17T03:16:40.947 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55213.log: /var/log/ceph/ceph-client.admin.56394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47393.log 2024-09-17T03:16:40.947 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55213.log.gz 2024-09-17T03:16:40.947 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44035.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56394.log.gz 2024-09-17T03:16:40.947 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43120.log 2024-09-17T03:16:40.947 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44035.log.gz 2024-09-17T03:16:40.948 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.47393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48112.log 2024-09-17T03:16:40.948 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47393.log.gz 2024-09-17T03:16:40.948 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36925.log 2024-09-17T03:16:40.948 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43120.log.gz 2024-09-17T03:16:40.948 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.48112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55171.log 2024-09-17T03:16:40.948 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48112.log.gz 2024-09-17T03:16:40.948 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43699.log 2024-09-17T03:16:40.948 INFO:teuthology.orchestra.run.smithi087.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.36925.log.gz 2024-09-17T03:16:40.949 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46904.log 2024-09-17T03:16:40.949 INFO:teuthology.orchestra.run.smithi087.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55171.log.gz 2024-09-17T03:16:40.949 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45337.log 2024-09-17T03:16:40.949 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43699.log.gz 2024-09-17T03:16:40.949 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.46904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48672.log 2024-09-17T03:16:40.949 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46904.log.gz 2024-09-17T03:16:40.949 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51189.log 2024-09-17T03:16:40.949 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45337.log.gz 2024-09-17T03:16:40.949 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.48672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55297.log 2024-09-17T03:16:40.950 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48672.log.gz 2024-09-17T03:16:40.950 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37010.log 2024-09-17T03:16:40.950 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51189.log: /var/log/ceph/ceph-client.admin.55297.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-17T03:16:40.950 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51189.log.gz 2024-09-17T03:16:40.950 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55297.log.gz 2024-09-17T03:16:40.950 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49180.log 2024-09-17T03:16:40.950 INFO:teuthology.orchestra.run.smithi087.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.37010.log.gz 2024-09-17T03:16:40.950 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43262.log 2024-09-17T03:16:40.951 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.49180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37061.log 2024-09-17T03:16:40.951 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48196.log 2024-09-17T03:16:40.951 INFO:teuthology.orchestra.run.smithi087.stderr: 91.1% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49180.log.gz 2024-09-17T03:16:40.951 INFO:teuthology.orchestra.run.smithi087.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43262.log.gz 2024-09-17T03:16:40.951 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44831.log 2024-09-17T03:16:40.951 INFO:teuthology.orchestra.run.smithi087.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.37061.log.gz 2024-09-17T03:16:40.951 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.48196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51984.log 2024-09-17T03:16:40.951 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44831.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48196.log.gz 2024-09-17T03:16:40.951 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53019.log 2024-09-17T03:16:40.952 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44831.log.gz 2024-09-17T03:16:40.952 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47512.log 2024-09-17T03:16:40.952 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51984.log.gz 2024-09-17T03:16:40.952 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55932.log 2024-09-17T03:16:40.952 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.47512.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53019.log.gz 2024-09-17T03:16:40.952 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48630.log 2024-09-17T03:16:40.952 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47512.log.gz 2024-09-17T03:16:40.952 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47848.log 2024-09-17T03:16:40.953 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55932.log: /var/log/ceph/ceph-client.admin.48630.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55932.log.gz 2024-09-17T03:16:40.953 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53730.log 2024-09-17T03:16:40.953 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48630.log.gz 2024-09-17T03:16:40.953 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.47848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44077.log 2024-09-17T03:16:40.953 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47848.log.gz 2024-09-17T03:16:40.953 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55339.log 2024-09-17T03:16:40.953 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51324.log 2024-09-17T03:16:40.953 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53730.log.gz 2024-09-17T03:16:40.954 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44077.log: /var/log/ceph/ceph-client.admin.55339.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44077.log.gz 2024-09-17T03:16:40.954 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49791.log 2024-09-17T03:16:40.954 INFO:teuthology.orchestra.run.smithi087.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55339.log.gz 2024-09-17T03:16:40.954 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36594.log 2024-09-17T03:16:40.954 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45295.log 2024-09-17T03:16:40.954 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51324.log.gz 2024-09-17T03:16:40.954 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49791.log: /var/log/ceph/ceph-client.admin.36594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48281.log 2024-09-17T03:16:40.954 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49791.log.gz 2024-09-17T03:16:40.954 INFO:teuthology.orchestra.run.smithi087.stderr: 76.1% -- replaced with /var/log/ceph/ceph-client.admin.36594.log.gz 2024-09-17T03:16:40.955 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47219.log 2024-09-17T03:16:40.955 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.48281.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45295.log.gz 2024-09-17T03:16:40.955 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44250.log 2024-09-17T03:16:40.955 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48281.log.gz 2024-09-17T03:16:40.955 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.47219.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-17T03:16:40.955 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51587.log 2024-09-17T03:16:40.955 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47219.log.gz 2024-09-17T03:16:40.955 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44250.log: /var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48844.log 2024-09-17T03:16:40.956 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44250.log.gz 2024-09-17T03:16:40.956 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52068.log 2024-09-17T03:16:40.956 INFO:teuthology.orchestra.run.smithi087.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.51587.log.gz 2024-09-17T03:16:40.956 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.48844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36425.log 2024-09-17T03:16:40.956 INFO:teuthology.orchestra.run.smithi087.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.48844.log.gz 2024-09-17T03:16:40.956 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45169.log 2024-09-17T03:16:40.956 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52068.log.gz 2024-09-17T03:16:40.956 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50604.log 2024-09-17T03:16:40.956 INFO:teuthology.orchestra.run.smithi087.stderr: 76.6% -- replaced with /var/log/ceph/ceph-client.admin.36425.log.gz 2024-09-17T03:16:40.957 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52683.log 2024-09-17T03:16:40.957 INFO:teuthology.orchestra.run.smithi087.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.45169.log.gz 2024-09-17T03:16:40.957 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44246.log 2024-09-17T03:16:40.957 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50604.log.gz 2024-09-17T03:16:40.957 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47596.log 2024-09-17T03:16:40.957 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52683.log.gz 2024-09-17T03:16:40.957 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56604.log 2024-09-17T03:16:40.957 INFO:teuthology.orchestra.run.smithi087.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.44246.log.gz 2024-09-17T03:16:40.958 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.47596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46802.log 2024-09-17T03:16:40.958 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47596.log.gz 2024-09-17T03:16:40.958 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.56604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43220.log 2024-09-17T03:16:40.958 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56604.log.gz 2024-09-17T03:16:40.958 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.46802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48537.log 2024-09-17T03:16:40.958 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46802.log.gz 2024-09-17T03:16:40.958 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43220.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-17T03:16:40.958 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43220.log.gz 2024-09-17T03:16:40.959 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.48537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56352.log 2024-09-17T03:16:40.959 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48537.log.gz 2024-09-17T03:16:40.959 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54357.log 2024-09-17T03:16:40.959 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.56352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53187.log 2024-09-17T03:16:40.959 INFO:teuthology.orchestra.run.smithi087.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56352.log.gz 2024-09-17T03:16:40.959 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49012.log 2024-09-17T03:16:40.959 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54357.log.gz 2024-09-17T03:16:40.959 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55591.log 2024-09-17T03:16:40.959 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53187.log: /var/log/ceph/ceph-client.admin.49012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51816.log 2024-09-17T03:16:40.960 INFO:teuthology.orchestra.run.smithi087.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.49012.log.gz 2024-09-17T03:16:40.960 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55591.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53187.log.gz 2024-09-17T03:16:40.960 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55591.log.gz 2024-09-17T03:16:40.960 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48154.log 2024-09-17T03:16:40.960 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52110.log 2024-09-17T03:16:40.960 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51816.log.gz 2024-09-17T03:16:40.960 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.48154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50339.log 2024-09-17T03:16:40.960 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48154.log.gz 2024-09-17T03:16:40.961 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56562.log 2024-09-17T03:16:40.961 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52110.log: /var/log/ceph/ceph-client.0.50339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52493.log 2024-09-17T03:16:40.961 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.56562.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52110.log.gz 2024-09-17T03:16:40.961 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56562.log.gz 2024-09-17T03:16:40.961 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55381.log 2024-09-17T03:16:40.961 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43909.log 2024-09-17T03:16:40.961 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52493.log: 90.7% -- replaced with /var/log/ceph/ceph-client.0.50339.log.gz 2024-09-17T03:16:40.961 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55381.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52493.log.gz 2024-09-17T03:16:40.962 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55381.log.gz 2024-09-17T03:16:40.962 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-17T03:16:40.962 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51774.log 2024-09-17T03:16:40.962 INFO:teuthology.orchestra.run.smithi087.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43909.log.gz 2024-09-17T03:16:40.962 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43867.log 2024-09-17T03:16:40.962 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56688.log 2024-09-17T03:16:40.962 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51774.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.51774.log.gz 2024-09-17T03:16:40.962 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43867.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43867.log.gz 2024-09-17T03:16:40.962 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36538.log 2024-09-17T03:16:40.963 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.56688.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.56688.log.gz 2024-09-17T03:16:40.963 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46760.log 2024-09-17T03:16:40.963 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36538.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-17T03:16:40.963 INFO:teuthology.orchestra.run.smithi087.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36538.log.gz 2024-09-17T03:16:40.963 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52935.log 2024-09-17T03:16:40.963 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.46760.log: /var/log/ceph/ceph.log: 90.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-17T03:16:40.963 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46760.log.gz 2024-09-17T03:16:40.963 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47806.log 2024-09-17T03:16:40.964 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52935.log: 85.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-17T03:16:40.964 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52893.log 2024-09-17T03:16:40.964 INFO:teuthology.orchestra.run.smithi087.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-17T03:16:40.964 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52935.log.gz 2024-09-17T03:16:40.964 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48970.log 2024-09-17T03:16:40.965 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.47806.log: /var/log/ceph/ceph-client.admin.52893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55255.log 2024-09-17T03:16:40.965 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47806.log.gz 2024-09-17T03:16:40.968 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52893.log.gz 2024-09-17T03:16:40.968 INFO:teuthology.orchestra.run.smithi087.stderr:gzip/var/log/ceph/ceph-client.admin.48970.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.52409.log 2024-09-17T03:16:40.968 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55255.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48970.log.gz 2024-09-17T03:16:40.968 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49138.log 2024-09-17T03:16:40.969 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55255.log.gz 2024-09-17T03:16:40.969 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55507.log 2024-09-17T03:16:40.969 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52409.log.gz 2024-09-17T03:16:40.969 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.49138.log: -- /var/log/ceph/ceph-client.admin.44829.log 2024-09-17T03:16:40.969 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55507.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49138.log.gz 2024-09-17T03:16:40.969 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45379.log 2024-09-17T03:16:40.969 INFO:teuthology.orchestra.run.smithi087.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55507.log.gz 2024-09-17T03:16:40.969 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53061.log 2024-09-17T03:16:40.970 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44929.log 2024-09-17T03:16:40.970 INFO:teuthology.orchestra.run.smithi087.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44829.log.gz 2024-09-17T03:16:40.970 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.45379.log: /var/log/ceph/ceph-client.admin.53061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48886.log 2024-09-17T03:16:40.970 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45379.log.gz 2024-09-17T03:16:40.970 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44929.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53061.log.gz 2024-09-17T03:16:40.970 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48407.log 2024-09-17T03:16:40.970 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44929.log.gz 2024-09-17T03:16:40.970 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43783.log 2024-09-17T03:16:40.971 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.48886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37369.log 2024-09-17T03:16:40.971 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.48407.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48886.log.gz 2024-09-17T03:16:40.971 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43783.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48407.log.gz 2024-09-17T03:16:40.971 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49707.log 2024-09-17T03:16:40.971 INFO:teuthology.orchestra.run.smithi087.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.43783.log.gz 2024-09-17T03:16:40.971 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46946.log 2024-09-17T03:16:40.971 INFO:teuthology.orchestra.run.smithi087.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.37369.log.gz 2024-09-17T03:16:40.971 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56520.log 2024-09-17T03:16:40.972 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.46946.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49707.log.gz 2024-09-17T03:16:40.972 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53295.log 2024-09-17T03:16:40.972 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46946.log.gz 2024-09-17T03:16:40.972 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.56520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47261.log 2024-09-17T03:16:40.972 INFO:teuthology.orchestra.run.smithi087.stderr:gzip 86.8% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.56520.log.gz -- 2024-09-17T03:16:40.972 INFO:teuthology.orchestra.run.smithi087.stderr: /var/log/ceph/ceph-client.admin.53229.log 2024-09-17T03:16:40.972 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53295.log: /var/log/ceph/ceph-client.admin.47261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44581.log 2024-09-17T03:16:40.972 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53295.log.gz 2024-09-17T03:16:40.973 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47261.log.gz 2024-09-17T03:16:40.973 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54415.log 2024-09-17T03:16:40.973 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51713.log 2024-09-17T03:16:40.973 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44581.log.gz 2024-09-17T03:16:40.973 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54415.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53229.log.gz 2024-09-17T03:16:40.973 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52247.log 2024-09-17T03:16:40.973 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54415.log.gz 2024-09-17T03:16:40.973 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53555.log 2024-09-17T03:16:40.974 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56016.log 2024-09-17T03:16:40.974 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51713.log.gz 2024-09-17T03:16:40.974 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52247.log: /var/log/ceph/ceph-client.0.53555.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52247.log.gz 2024-09-17T03:16:40.974 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44830.log 2024-09-17T03:16:40.974 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.56016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50723.log 2024-09-17T03:16:40.974 INFO:teuthology.orchestra.run.smithi087.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56016.log.gz 2024-09-17T03:16:40.974 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47638.log 2024-09-17T03:16:40.975 INFO:teuthology.orchestra.run.smithi087.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.44830.log.gz 2024-09-17T03:16:40.975 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49663.log 2024-09-17T03:16:40.975 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50723.log.gz 2024-09-17T03:16:40.975 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55633.log 2024-09-17T03:16:40.975 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.47638.log: 91.4% -- replaced with /var/log/ceph/ceph-client.0.53555.log.gz 2024-09-17T03:16:40.975 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49663.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.47638.log.gz 2024-09-17T03:16:40.975 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47722.log 2024-09-17T03:16:40.975 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49663.log.gz 2024-09-17T03:16:40.976 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55890.log 2024-09-17T03:16:40.976 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55633.log.gz 2024-09-17T03:16:40.976 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.47722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55549.log 2024-09-17T03:16:40.976 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47722.log.gz 2024-09-17T03:16:40.976 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37055.log 2024-09-17T03:16:40.976 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55890.log: /var/log/ceph/ceph-client.admin.55549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54183.log 2024-09-17T03:16:40.976 INFO:teuthology.orchestra.run.smithi087.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55890.log.gz 2024-09-17T03:16:40.977 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55549.log.gz 2024-09-17T03:16:40.977 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49054.log 2024-09-17T03:16:40.977 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.37055.log: /var/log/ceph/ceph-client.0.54183.log: gzip 85.8% -- replaced with /var/log/ceph/ceph-client.admin.37055.log.gz -5 2024-09-17T03:16:40.977 INFO:teuthology.orchestra.run.smithi087.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53846.log 2024-09-17T03:16:40.977 INFO:teuthology.orchestra.run.smithi087.stderr:gzip/var/log/ceph/ceph-client.admin.49054.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44592.log 2024-09-17T03:16:40.977 INFO:teuthology.orchestra.run.smithi087.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49054.log.gz 2024-09-17T03:16:40.977 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54473.log 2024-09-17T03:16:40.978 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53846.log.gz 2024-09-17T03:16:40.978 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36751.log 2024-09-17T03:16:40.978 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44592.log: 89.4% -- replaced with /var/log/ceph/ceph-client.0.54183.log.gz 2024-09-17T03:16:40.978 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54473.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.44592.log.gz 2024-09-17T03:16:40.978 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48365.log 2024-09-17T03:16:40.978 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54473.log.gz 2024-09-17T03:16:40.978 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43657.log 2024-09-17T03:16:40.979 INFO:teuthology.orchestra.run.smithi087.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36751.log.gz 2024-09-17T03:16:40.979 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.48365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44588.log 2024-09-17T03:16:40.979 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43657.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48365.log.gz 2024-09-17T03:16:40.980 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47335.log 2024-09-17T03:16:40.980 INFO:teuthology.orchestra.run.smithi087.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43657.log.gz 2024-09-17T03:16:40.980 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44930.log 2024-09-17T03:16:40.981 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44588.log.gz 2024-09-17T03:16:40.981 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.47335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56729.log 2024-09-17T03:16:40.981 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47335.log.gz 2024-09-17T03:16:40.981 INFO:teuthology.orchestra.run.smithi087.stderr:gzip/var/log/ceph/ceph-client.admin.44930.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44247.log 2024-09-17T03:16:40.981 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.56729.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44930.log.gz 2024-09-17T03:16:40.981 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43527.log 2024-09-17T03:16:40.982 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56729.log.gz 2024-09-17T03:16:40.982 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44252.log 2024-09-17T03:16:40.982 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43527.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44247.log.gz 2024-09-17T03:16:40.982 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51942.log 2024-09-17T03:16:40.982 INFO:teuthology.orchestra.run.smithi087.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43527.log.gz 2024-09-17T03:16:40.982 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.44252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44254.log 2024-09-17T03:16:40.982 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52977.log 2024-09-17T03:16:40.982 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44252.log.gz 2024-09-17T03:16:40.983 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51942.log: /var/log/ceph/ceph-client.admin.44254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56478.log 2024-09-17T03:16:40.983 INFO:teuthology.orchestra.run.smithi087.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51942.log.gz 2024-09-17T03:16:40.983 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44254.log.gz 2024-09-17T03:16:40.983 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36978.log 2024-09-17T03:16:40.983 INFO:teuthology.orchestra.run.smithi087.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52977.log.gz 2024-09-17T03:16:40.983 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.56478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46988.log 2024-09-17T03:16:40.983 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36978.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56478.log.gz 2024-09-17T03:16:40.984 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52581.log 2024-09-17T03:16:40.984 INFO:teuthology.orchestra.run.smithi087.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.36978.log.gz 2024-09-17T03:16:40.984 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.46988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51461.log 2024-09-17T03:16:40.984 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46988.log.gz 2024-09-17T03:16:40.984 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54550.log 2024-09-17T03:16:40.984 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52581.log: /var/log/ceph/ceph-client.admin.51461.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.52581.log.gz -5 2024-09-17T03:16:40.984 INFO:teuthology.orchestra.run.smithi087.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48802.log 2024-09-17T03:16:40.984 INFO:teuthology.orchestra.run.smithi087.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.51461.log.gz 2024-09-17T03:16:40.985 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.54550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47554.log 2024-09-17T03:16:40.985 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54550.log.gz 2024-09-17T03:16:40.985 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.48802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52725.log 2024-09-17T03:16:40.985 INFO:teuthology.orchestra.run.smithi087.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48802.log.gz 2024-09-17T03:16:40.985 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.47554.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-17T03:16:40.985 INFO:teuthology.orchestra.run.smithi087.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47554.log.gz 2024-09-17T03:16:40.986 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.52725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36975.log 2024-09-17T03:16:40.986 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52725.log.gz 2024-09-17T03:16:40.986 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43615.log 2024-09-17T03:16:40.986 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.36975.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-17T03:16:40.987 INFO:teuthology.orchestra.run.smithi087.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.36975.log.gz 2024-09-17T03:16:40.987 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.43615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43392.log 2024-09-17T03:16:40.987 INFO:teuthology.orchestra.run.smithi087.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43615.log.gz 2024-09-17T03:16:40.987 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51858.log 2024-09-17T03:16:40.987 INFO:teuthology.orchestra.run.smithi087.stderr:gzip/var/log/ceph/ceph-client.admin.43392.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50079.log 2024-09-17T03:16:40.987 INFO:teuthology.orchestra.run.smithi087.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43392.log.gz 2024-09-17T03:16:40.988 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.51858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49885.log 2024-09-17T03:16:40.988 INFO:teuthology.orchestra.run.smithi087.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51858.log.gz 2024-09-17T03:16:40.988 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.50079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51629.log 2024-09-17T03:16:40.988 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50079.log.gz 2024-09-17T03:16:40.988 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53788.log 2024-09-17T03:16:40.988 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.49885.log: /var/log/ceph/ceph-client.admin.51629.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49885.log.gz 2024-09-17T03:16:40.988 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51629.log.gz 2024-09-17T03:16:40.989 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47764.log 2024-09-17T03:16:40.989 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.53788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55078.log 2024-09-17T03:16:40.989 INFO:teuthology.orchestra.run.smithi087.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53788.log.gz 2024-09-17T03:16:40.989 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.47764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46566.log 2024-09-17T03:16:40.989 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47764.log.gz 2024-09-17T03:16:40.990 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.55078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55974.log 2024-09-17T03:16:40.990 INFO:teuthology.orchestra.run.smithi087.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55078.log.gz 2024-09-17T03:16:40.990 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.46566.log: /var/log/ceph/ceph-client.admin.55974.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46566.log.gz 2024-09-17T03:16:40.990 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55974.log.gz 2024-09-17T03:16:40.991 INFO:teuthology.orchestra.run.smithi087.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-17T03:16:40.992 INFO:teuthology.orchestra.run.smithi149.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-17T03:16:41.011 INFO:teuthology.orchestra.run.smithi149.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-17T03:16:41.013 INFO:teuthology.orchestra.run.smithi087.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-17T03:16:41.018 INFO:teuthology.orchestra.run.smithi149.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-17T03:16:41.032 INFO:teuthology.orchestra.run.smithi149.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-17T03:16:41.042 INFO:teuthology.orchestra.run.smithi149.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-17T03:16:41.048 INFO:teuthology.orchestra.run.smithi087.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-17T03:16:41.067 INFO:teuthology.orchestra.run.smithi149.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-17T03:16:41.069 INFO:teuthology.orchestra.run.smithi149.stderr: 2024-09-17T03:16:41.069 INFO:teuthology.orchestra.run.smithi149.stderr:real 0m0.209s 2024-09-17T03:16:41.069 INFO:teuthology.orchestra.run.smithi149.stderr:user 0m0.958s 2024-09-17T03:16:41.070 INFO:teuthology.orchestra.run.smithi149.stderr:sys 0m0.058s 2024-09-17T03:16:41.073 INFO:teuthology.orchestra.run.smithi087.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-17T03:16:41.087 INFO:teuthology.orchestra.run.smithi087.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-17T03:16:41.112 INFO:teuthology.orchestra.run.smithi087.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-17T03:16:41.113 INFO:teuthology.orchestra.run.smithi087.stderr: 2024-09-17T03:16:41.114 INFO:teuthology.orchestra.run.smithi087.stderr:real 0m0.279s 2024-09-17T03:16:41.114 INFO:teuthology.orchestra.run.smithi087.stderr:user 0m1.048s 2024-09-17T03:16:41.114 INFO:teuthology.orchestra.run.smithi087.stderr:sys 0m0.204s 2024-09-17T03:16:41.114 INFO:tasks.ceph:Archiving logs... 2024-09-17T03:16:41.114 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908414/remote/smithi087/log 2024-09-17T03:16:41.116 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T03:16:41.539 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908414/remote/smithi149/log 2024-09-17T03:16:41.540 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-17T03:16:41.710 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-17T03:16:41.761 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-17T03:16:41.761 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T03:16:41.764 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-17T03:16:41.835 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-17T03:16:41.848 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-17T03:16:41.848 DEBUG:teuthology.orchestra.run.smithi087:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T03:16:41.851 DEBUG:teuthology.orchestra.run.smithi149:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-17T03:16:41.864 INFO:teuthology.orchestra.run.smithi087.stderr:bash: line 1: ntpq: command not found 2024-09-17T03:16:41.865 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-09-17T03:16:41.867 INFO:teuthology.orchestra.run.smithi087.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T03:16:41.867 INFO:teuthology.orchestra.run.smithi087.stdout:=============================================================================== 2024-09-17T03:16:41.867 INFO:teuthology.orchestra.run.smithi087.stdout:^* hv01.front.sepia.ceph.com 3 6 377 50 +1972us[+2403us] +/- 40ms 2024-09-17T03:16:41.867 INFO:teuthology.orchestra.run.smithi087.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 50 -928us[ -497us] +/- 76ms 2024-09-17T03:16:41.868 INFO:teuthology.orchestra.run.smithi087.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 -2848us[-2848us] +/- 82ms 2024-09-17T03:16:41.868 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T03:16:41.869 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-17T03:16:41.869 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-09-17T03:16:41.870 INFO:teuthology.orchestra.run.smithi149.stdout:^* hv01.front.sepia.ceph.com 3 6 377 51 +1975us[+2428us] +/- 40ms 2024-09-17T03:16:41.870 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 51 -909us[ -909us] +/- 76ms 2024-09-17T03:16:41.870 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 51 -2834us[-2380us] +/- 82ms 2024-09-17T03:16:41.870 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-17T03:16:41.870 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-17T03:16:41.879 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-17T03:16:41.879 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-17T03:16:41.922 DEBUG:teuthology.orchestra.run.smithi087:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T03:16:41.993 DEBUG:teuthology.orchestra.run.smithi149:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-17T03:16:42.056 DEBUG:teuthology.orchestra.run.smithi087:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T03:16:42.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:16:42.085 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-17T03:16:42.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:16:42.114 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-17T03:16:42.123 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-17T03:16:42.157 INFO:teuthology.task.internal:Duration was 548.057379 seconds 2024-09-17T03:16:42.158 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-17T03:16:42.201 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-17T03:16:42.201 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T03:16:42.203 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-17T03:16:42.239 INFO:teuthology.orchestra.run.smithi087.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T03:16:42.242 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-17T03:16:42.585 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-17T03:16:42.586 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi087.front.sepia.ceph.com 2024-09-17T03:16:42.586 DEBUG:teuthology.orchestra.run.smithi087:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T03:16:42.645 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi149.front.sepia.ceph.com 2024-09-17T03:16:42.646 DEBUG:teuthology.orchestra.run.smithi149:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-17T03:16:42.679 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-17T03:16:42.680 DEBUG:teuthology.orchestra.run.smithi087:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T03:16:42.688 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-17T03:16:42.746 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-17T03:16:42.747 DEBUG:teuthology.orchestra.run.smithi087:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T03:16:42.755 DEBUG:teuthology.orchestra.run.smithi149:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-17T03:16:42.889 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-17T03:16:42.899 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-17T03:16:42.900 DEBUG:teuthology.orchestra.run.smithi087:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T03:16:42.925 DEBUG:teuthology.orchestra.run.smithi149:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-17T03:16:42.950 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-17T03:16:42.960 DEBUG:teuthology.orchestra.run.smithi087:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T03:16:42.968 DEBUG:teuthology.orchestra.run.smithi149:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-17T03:16:42.989 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern = core 2024-09-17T03:16:43.014 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = core 2024-09-17T03:16:43.027 DEBUG:teuthology.orchestra.run.smithi087:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T03:16:43.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:16:43.057 DEBUG:teuthology.orchestra.run.smithi149:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-17T03:16:43.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-17T03:16:43.081 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-17T03:16:43.093 INFO:teuthology.task.internal:Transferring archived files... 2024-09-17T03:16:43.094 DEBUG:teuthology.misc:Transferring archived files from smithi087:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908414/remote/smithi087 2024-09-17T03:16:43.094 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T03:16:43.098 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-17T03:16:43.098 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-09-17T03:16:43.098 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-09-17T03:16:43.099 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-09-17T03:16:43.133 DEBUG:teuthology.misc:Transferring archived files from smithi149:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-17_02:09:51-fs-wip-pdonnell-testing-20240916.200549-debug-distro-default-smithi/7908414/remote/smithi149 2024-09-17T03:16:43.133 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-17T03:16:43.166 INFO:teuthology.task.internal:Removing archive directory... 2024-09-17T03:16:43.166 DEBUG:teuthology.orchestra.run.smithi087:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T03:16:43.170 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-17T03:16:43.218 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-17T03:16:43.229 INFO:teuthology.task.internal:Not uploading archives. 2024-09-17T03:16:43.229 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-17T03:16:43.257 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-17T03:16:43.257 DEBUG:teuthology.orchestra.run.smithi087:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T03:16:43.260 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-17T03:16:43.273 INFO:teuthology.orchestra.run.smithi087.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 03:16 /home/ubuntu/cephtest 2024-09-17T03:16:43.274 INFO:teuthology.orchestra.run.smithi149.stdout: 265613 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 17 03:16 /home/ubuntu/cephtest 2024-09-17T03:16:43.276 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-17T03:16:43.299 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-17T03:16:43.385 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/pool-perm} duration: 548.0573785305023 failure_reason: 'Test failure: test_pool_perm (tasks.cephfs.test_pool_perm.TestPoolPerm)' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=18421a23d5bc45bf8002ca7e50a51029 status: fail success: false 2024-09-17T03:16:43.385 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-17T03:16:43.470 INFO:teuthology.run:FAIL